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CHAPTER  1 


INTRODUCTION 


1 .  General  Remarks 

The  problem  of  optimal  control  of  uncertain  systems  has  tradi¬ 
tionally  been  treated  in  a  stochastic  framework  in  the  sence  that  the  un¬ 
certain  quantities  are  modeled  as  random  vectors  and  random  processes 
with  statistical  properties  which  are  assumed  known.  The  controller 
selected  is  the  one  for  which  the  expected  value  of  a  suitable  cost  func¬ 
tional  is  minimized.  In  this  framework  some  mathematically  elegant 
results  have  been  obtained,  notable  cases  being  the  separation  theorem 
for  a  linear  system, linear  measurements  and  quadratic  cost  functionalP^’ 
(Gl),  (Sul)  an{j  tke  8eparation  theorem  for  a  linear  system,  linear  mea¬ 
surements,  Gaussian  disturbances  and  nonquadratic  cost  functional^*^’ 
(W°3)  Specification  of  the  a  priori  statistics  of  all  the  uncertain  quantities 
involved  must  be  made  in  any  such  problem.  In  many  practical  situations 
however  these  statistics  are  not  available,  and  cannot  be  obtained  either 
because  of  physical  constraints  or  due  to  prohibitive  cost.  In  such  cases 
however  the  designer  may  have  information  of  less  detailed  structure  con¬ 
cerning  the  uncertain  quantities,  such  as  for  instance  bounds  on  the  mag¬ 
nitude  or  energy  of  the  uncertain  quantities.  In  other  words  the  designer 
may  be  given  a  set  where  the  uncertain  quantities  are  known  to  belong. 

A  possible  design  approach  under  these  circumstances  would  then  be  to 
select  the  controller  from  some  admissible  class  which  performs  best 
when  the  uncertain  quantities  assume  their  worst  possible  values  within 
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the  given  set.  In  its  simplest  form  the  corresponding  decision  problem 
is  described  by  a  triplet  (U.Q,  J),  where  U  is  the  set  of  controllers  under 
consideration,  Q  is  the  set  in  which  the  uncertain  quantities  are  known 
to  belong  and  J:UxQ  -*  [  -oo,  +oo]  is  a  given  cost  function.  The  objective 
is  to  find 

J  =  inf  sup  J(u,q)  (1.1) 

ue  U  qeQ 

and,  if  it  exists,  vhe  minimizing  controller  u  in  U. 

Problems  of  the  general  form  of  equation  (1.1)  can  also  arise  in 
the  context  of  other  situations,  In  some  cases  the  nature  of  the  problem 
calls  for  a  pessimistic  or  worst  case  approach  such  as  when  specified 

r 

tolerances  must  be  met  with  certainty.  For  example  in  a  chemical  process 
control  problem  it  may  be  necessary  to  gu’arantee  that  the  state  will  stay  in 
a  specified  region  of  the  state  space,  or  equi  v^lently  avoid  a  critical  region 
of  the  state  space  where  process  instability  may'  occur.  In  other  cases  a 
worst  case  analysis  is  performed  in  order  to  provio*e  a  comparison  with 
the  performance  of  a  design  adopted  on  the  basiB  of  oth^r  considerations. 

Optimal  uncertain  control  problems  that  can  be  redufctd  to  the  form 
of  equation  (1.1)  are  referred  to  as  Minimax  Control  Problems  and  are 
the  object  of  study  of  this  thesis. 

The  modelling  of  uncertainties  as  quantities  that  are  unknown  except 

that  they  belong  to  prescribed  sets  has  received  attention  before,  dating  to 

(Wal ) 

Wald's  statistical  decision  theory.  1  In  the  context  of  Wald's  theo.  > 
the  decision  problem  (U,Q,  J)  mentioned  earlier  is  viewed  as  a  game  against 
Nature  and  a  saddle  point  of  this  game  in  (possibly)  randomized  strategies 
is  sought.  Whenever  a  saddle  point  in  pure  strateg.' , ..y  exists,  i.  e. ,  whenever 
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inf  sup  J(u,  q)  =  sup  inf  J(u,  q)  (1.2) 

ueU  qeQ  qcQ  ueU 

Wald's  approach  is  equivalent  to  the  worst  case  approach.  When  however 

the  equality  (1.2)  does  not  hold  Wald's  theory  recommends  randomization 

in  the  spaces  of  strategies  U  and  Q,  and  the  worst  case  viewpoint  is  lost. 

(Swl ) 

Wald's  theory  was  applied  by  Sworder'  to  discrete-time  control  systems 
with  limited  success  since  randomization  within  the  admissible  set  of  con¬ 
trollers  was  not  considered  appealing  from  the  practical  viewpoint  of  an 
engineer. 

The  consideration  of  the  minimax  approach  to  the  optimal  control 

of  discrete-time  uncertain  systems  without  the  randomization  suggested 

fFl )  fF2) 

by  Wald's  theory  was  recommended  by  Feldbaum'  ’’  '  '  and  systemati¬ 

cally  studied  by  Witsenhausen!^^’ 

Problems  of  system  state  estimation  for  the  case  where  the  un¬ 
certain  quantities  are  described  by  their  membership  in  given  sets  have 
also  been  considered  by  Witsenhausen,^^’  Schweppe^^’ 

and  others. Such  problems,  though  important  in  their  own  right, 
arise  in  connection  with  minimax  control  problems  for  which  the  controller 
has  available  only  a  noise -corrupted  measurement  of  an  output  of  the  sys¬ 
tem  rather  than  an  exact  measurement  of  the  system  state.  Although  the 
emphasis  in  this  thesis  is  in  the  feedback  control  of  uncertain  systems, 
some  state  estimation  problems  will  also  be  considered  which  have  a  direct 
relation  to  feedback  control  problems.  In  the  next  section  we  shall  state 
the  basic  problem  considered  in  the  thesis  and  outline  the  general  approach 
which  we  will  adopt  towards  its  solution. 
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2.  The  Basic  Problem 

The  objective  of  this  thesis  is  the  study  of  the  following  problem: 

Problem  1.1:  Given  is  the  discrete-time  dynamic  system 

Xj^j  -  fj^(Xj^»  u^,  w^)  *  k  =  0,  1,  ,  N-l  (1.3) 

where  x^eRn,  k  =  0,  1,  . . ,  N  is  the  state  vector,  u^fRm,  k  =  0,  1,  ,  N-l, 

is  the  control  vector,  w^eR1",  k  =  0,  1,  . . .,  N-l,  is  the  input  disturbance 
vector,  and  f^:Rn  x  Rm  x  Rr  -*•  Rn  are  known  functions. 

Available  to  the  controller  are  measurements  of  the  form 

‘f 

zk  =  W  Vk>*  •  k=l,2 . N-l  (1.4) 

where,  for  all  k  =  1,2,  ...,N-1,  z^cR8  is  the  measurement  vector,  v^eRp 

is  the  measurement  noise  vector,  and  h^:Rn  x  Rp  -*  Rs  are  known  functions. 

The  uncertain  quantities  lumped  in  a  vector  qcRn+Nr+^N~ 1 

q  =  (XQ,  Wq*  wj*  •  •  • »  wn_i»  v\,  v2*  * '  *  *  vN-l)*  O*5) 

are  known  to  belong  to  a  given  subset  Q  of  pn+Nr+(N-l)p 

qcQ  (1.6) 


We  restrict  attention  to  this  form  of  measurement  equation  without 
loss  of  generality.  A  measurement  equation  containing  the  control 
vector  explicitly 


*k  *  *k(Wrvk> 

can  be  reduced  to  the  form  (1.4)  by  introducing  additional  state  variables 
through  the  equation 
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Attention  is  restricted  to  control  laws  of  the  form 
:  Rk(s+m)  -  Rm,  k  =  0,  1,  . . . ,  N-l 

taking  values 

=  JAj^(2|iZ2,,**,zk'  ^o*  ^1'  "  ‘ '  uk- 1  ^  k  =  ^ . N-l 

where  is  interpreted  as  a  constant  vector  (p.Q  =  uq).  It  is  required  to 
find  (if  it  exists)  the  control  law  in  this  class  for  which  the  cost  functional 

J(H  .  Up  •  •»,an-1)  =  8UP  FIx1»x2,  .  .,xN,p  .Pjtzj.u 

q«Q 

/  V,  (1 . 7) 

^N-l^l*  *  *  *  ’  uN-2^ 

I 

is  minimized,  subject  to  the  system  and  measurement  equation  constraints 
(1.3),  (1.4)  and  where  the  function  F:RN^n+m^  -*  (-oo,  ao)  is  given. 

It  should  be  noted  that  in  the  statement  of  the  above  problem  we  take 
into  account  implicitly  the  presence  of  state  and  control  constraints,  since 
we  allow  the  function  F  in  the  cost  functional  (1.7)  to  take  the  value  oo.  We 
simply  specify  that  the  function  F  takes  the  value  oo  whenever  some  constraint 
is  violated.  Thus,  for  example,  state  and  control  constraints  of  the  form 

x^cX^,  uk-icUk-r  w^ere  Xk’  ^k-1’  k  =  ^*2*',*N*  are  given  sets,  are 
accounted  for  by  adding  to  the  function  F  the  function 

N 

2  {6(x  |X.)  +  6[p._1(z1^.,ui_2)|U._1)} 

i_  j 

where  6 (y |  Y)  denotes  the  indicator  function  of  a  set  Y  (6 (y |  Y)  =  0  if  yeY, 

6(y|  Y)  =  oo  if  yrfY). 
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The  Problem  1.1  can,  in  principle,  be  solved  by  dynamic  program¬ 
ming,  and  the  appropriate  algorithm  will  be  presented  in  this  thesis.  How¬ 
ever  it  is  in  general  very  difficult  from  this  algorithm  to  characterize  ef¬ 
ficiently  the  optimal  controller  which  solves  Problem  1.1.  Thus  special 
cases  with  increased  structure  will  be  considered  in  order  to  obtain  addi¬ 
tional  results  related  to  the  characterization  of  the  optimal  controller  and 
in  order  to  gain  increased  understanding  into  the  structure  of  the  solution. 

One  of  the  major  difficulties  in  solving  the  general  Problem  1.  1 
results  from  the  fact  that  the  value  of  the  current  state  of  the  system  (1.  3) 
is  not  available  to  the  controller  but  instead  only  partial  information  is 
known  about  it  via  the  measurements  (1.4).  This  fact  results  in  that,  in 

l 

general,  the  optimal  control  law  will  be  a  function  of  all  the  prior  mea¬ 
surements,  i.e.,  in  general  the  controller  will  need  to  store  all  the  prior 
measurements  or,  possibly,  the  value  of  a  complicated  function  of  these 
measurements.  However,  as  in  the  corresponding  stochastic  situation, 
whenever  an  exact  measurement  of  the  current  state  is  available  to  the  con¬ 
troller,  i.e.,  in  equation  (1 . 4)  we  have 

hk(xk»  vk>  =  xk  (1'8) 

and  in  addition  the  input  disturbances  w^  are  individually  constrained  at  each 
time 

w.fW.C  Rr 
k  k 

and  the  function  F  in  equation  (1.7)  is  of  the  additive  form 

N 

F(x1,x2,  •  ••*N,u0,u1,  .  •  »un_j)  =  k^jgk*xk,uk-l* 
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then  it  can  be  shown  that  the  optimal  control  law  is  of  the  form  =  pi^(x^). 

In  other  words  the  control  law  need  only  be  a  function  of  the  current  state, 
with  a  substantial  simplification  resulting.  Alternatively  expressed,  under 
the  circumstances  described  above,  the  value  of  the  current  state  contains 
all  information  about  the  past  history  of  the  system  which  is  necessary  for 
the  specification  of  the  optimal  control. 

The  special  case  of  Problem  1.1  where  equation  (1.8)  holds  is  re¬ 
ferred  to  as  the  minimax  control  problem  with  perfect  state  information  and 
receives  considerable  attention  in  this  thesis.  A  large  part  of  the  thesis, 
Chapters  2  and  3,  are  devoted  to  problems  with  perfect  state  information. 
This  serves  a  double  purpose.  In  addition  to  studying  a  class  of  problems 
which  is  of  interest  in  its  own  right,  we  obtain  results  which  are  useful  for 
deriving  optimal  or  suboptimal  solutions  for  som§  minimax  control  problems 
with  imperfect  state  information.  This  is  true  in  particular  for  the  problem 
of  the  reachability  of  a  target  tube  which  will  be  considered  extensively  in 
the  thesis. 

3.  Contributions  and  Organization  of  the  Thesis 

The  problems  considered  in  this  thesis  can  be  divided  into  three 
broad  categories.  Minimax  control  problems  with  perfect  state  information 
are  considered  in  Chapters  2  and  3,  minimax  control  p-oblems  with  im¬ 
perfect  state  information  are  considered  in  Chapters  5  and  6,  and  state 
estimation  problems  are  examined  in  Chapter  4. 

In  Chapter  2  a  minimax  control  problem  with  perfect  state  information 
is  considered  for  the  case  of  a  linear  system  and  a  cost  functional  with  some 


I 
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convexity  properties.  This  problem  in  a  somewhat  less  general  form  was 

(Wl)  (W2) 

considered  first  by  Witsenhausen/  ”  '  1  Some  new  results  concerning 

existence  of  optimal  control  laws  are  obtained,  and  the  investigation  of 
necessary  conditions  for  optimality  is  carried  out  in  depth.  A  minimax 
principle  is  derived  for  this  problem  which  holds  however  only  under  some 
restrictive  assumptions.  When  specialized  to  the  case  of  a  deterministic 
optimal  control  problem  this  minimax  principle  yields  a  minimum  principle 
for  which  the  cost  functional  is  not  required  to  be  differentiable. 

In  Chapter  3  the  problem  of  reachability  of  a  target  tube  is  considered 
for  the  perfect  state  information  case.  This  problem  can  be  viewed  as  a 
special  case  of  the  problem  considered  in  Chapter  2.  Necessary  and  suf¬ 
ficient  conditions  for  the  existence  of  a  solution  are  obtained.  These  con¬ 
ditions  can  also  be  derived  with  little  effort  from  Witsenhausen's  results.^1 
(W21 

'  ’  In  addition  a  new  ellipsoidal  approximation  algorithm,  which  appears 

to  have  some  potential  for  practical  applications,  is  derived  and  its  properties 
are  investigated. 

In  Chapter  4  the  problem  of  the  system  state  estimation  is  examined 
for  a  set-membership  description  of  the  uncertainty.  Attention  is  restricted 
to  linear  systems  and  two  different  set-membership  descriptions  of  the  un¬ 
certainty,  the  cases  of  energy  constzaints  and  instantaneous  ellipsoidal  con¬ 
straints  on  the  uncertain  quantities.  Some  new  estimation  algorithms  are 
obtained  for  both  cases.  In  particular,  for  the  case  of  instantaneous  ellip¬ 
soidal  constraints  for  the  uncertain  quantities,  an  estimator  is  obtained  which 
offers  distinct  advantages  over  the  estimator  proposed  by  Schweppe.  ' 
Furthermore  we  use  a  new  approach  towards  the  solution  of  the  problem  which 
allows  us  to  treat  some  problems  not  considered  as  yet  in  the  literature  in¬ 
cluding  the  smoothing  problem. 
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In  Chapter  5  the  general  case  of  problem  1 . 1  is  examined  and  the 
dynamic  programming  algorithm  for  its  solution  is  developed.  This  algo¬ 
rithm  differs  in  its  form  and  is  more  general  than  the  algorithm  of 
Witsenhausen'  ’  although  the  same  basic  ideas  are  involved.  Subsequently 
the  notion  of  a  sufficiently  informative  function,  which  parallels  the  notion 
of  a  sufficient  statistic  of  stochastic  optimal  control*  is  formulated  for  the 
first  time.  Some  results  are  then  derived  which  illustrate  the  dual  function 
of  the  optimal  controller  as  an  estimator  and  an  actuator.  This  parallels 
the  dual  estimation-actuation  interpretation  of  the  function  of  the  optimal 
controller  in  the  analogous  problem  when  the  uncertainties  are  modeled  as 
random  vectors  or  stochastic  processes. 

Finally  in  Chapter  6  the  problem  of  the  reachability  of  a  target  tube 
with  imperfect  state  information  is  considered  for  the  case  of  a  linear  system. 
The  material  in  this  chapter  is  new.  For  the  special  case  of  energy  con¬ 
straints  on  the  uncertain  quantities  the  optimal  controller  is  completely 
characterized,  and  its  separation  in  an  estimator  and  an  actuator  is  explicitly 
demonstrated.  The  case  of  instantaneous  ellipsoidal  constraints  on  the  un¬ 
certain  quantities  is  also  considered,  and  a  suboptimal  algorithm  is  derived 
which  offers  some  practical  implementation  advantages. 

For  the  development  of  some  of  the  results  of  Chapter  2  it  is  nec- 

(R 1 ) 

essary  to  appeal  in  a  nontrivial  way  to  the  theory  of  convex  functions.' 

Since  portion^  of  this  theory  are  comparatively  recent  and  not  very  widely 
known,  the  required  results  have  been  summarized  in  Appendix  I.  It  should 
be  noted  that  this  theory  is  used  only  in  Chapter  2,  and  is  not  necessary  for 
the  developments  in  the  remainder  of  the  thesis. 


CHAPTER  2 


LINEAR  MINIMAX  CONTROL  PROBLEMS 
WITH  PERFECT  STATE  INFORMATION 

1 .  General  Remarks 

In  this  chapter  we  consider  a  minimax  control  problem  with  perfect 
state  information.  As  was  mentioned  in  the  previous  chapter  the  fact  that 
the  controller  has  available  at  each  time  a  perfect  measurement  of  the  sys¬ 
tem  state  results  in  a  substantial  simplification  in  the  solution  of  the  prob¬ 
lem.  For  example  the  dynamic  programming  algorithm,  which  is  the  basic 
method  for  solving  minimax  control  problems,  becomes  greatly  simplified 
for  this  case.  Furthermore,  in  this  chapter  we  make  some  additional  as¬ 
sumptions  which  enable  us  to  obtain  some  deeper  analytical  results.  We 
assume  that  the  dynamical  system  involved  is  linear,  and  that  the  cost  func¬ 
tional  has  some  convexity  properties.  This  will  allow  us  to  consider  in  de¬ 
tail  questions  of  existence  of  solutions  and  necessary  conditions  for  optimality. 
In  addition  it  will  be  shown  for  this  case  that  if  the  sets  where  the  input  dis¬ 
turbances  are  known  to  belong  are  polyhedra,  the  computational  requirements 
of  the  dynamic  programming  algorithm  can  be  further  significantly  reduced. 
The  results  mentioned  above  rely  heavily  on  the  additional  structure  of  lin¬ 
earity  for  the  system  and  convexity  for  the  cost  functional,  and  do  not  appear 
to  be  available  without  them.  In  this  way  the  problem  of  this  chapter  should 
be  considered  as  the  special  case  of  the  minimax  control  problem  1 .  1  which 
is  most  amenable  to  somewhat  deeper  analysis,  and  for  which  the  obtained 
results  are  considerably  stronger  than  in  the  general  case.  Yet  this  special 
case  is  sufficiently  general  to  be  of  interest  in  its  own  right,  and  the  cor- 
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responding  results  provide  insights  into  the  solution  of  other  more  gen¬ 
eral  minimax  control  problems. 

For  t^e  development  of  some  of  the  results  of  this  chapter  we  will 

need  to  draw  heavily  on  some  comparatively  recent  and  not  very  widely 

fR  1 ) 

known  results  of  'he  theory  of  convex  functions.1  '  The  related  theory 
has  been  outlined  in  Appendix  1  and  will  be  used  mainly  after  Section  3  of 
this  chapter.  This  theory  will  not  be  needed  later  in  the  thesis.  The 
reader  who  is  interested  in  subsequent  chapters  can  proceed  to  those  chapters 
after  section  3  without  loss  of  continuity. 

In  the  next  section  the  minimax  control  problem  of  this  chapter  will 
be  formulated  and  its  solution  by  dynamic  programming  will  be  shown  sub¬ 
sequently  in  Section  3.  In  Section  4  the  properties  of  the  dynamic  program¬ 
ming  algorithm  will  be  investigated  and  sufficient  conditions  for  existence  of 
optimal  control  laws  will  be  derived.  In  Section  5  necessary  conditions  for 
optimality  will  be  obtained.  In  particular  a  minimax  principle  is  proved 
which  however  holds  under  somewhat  restrictive  assumptions.  When  spe¬ 
cialized  to  deterministic  optimal  control  problems  this  minimax  principle 
yields  a  minimum  principle  for  which  the  cost  functional  is  not  assumed 
differentiable. 

2.  Problem  Formulation 

The  object  of  study  in  this  chapter  is  the  following  problem. 

Problem  2.1:  Consider  the  linear  discrete-time  dynamic  system: 

xk+l  =  Akxk  +  Bkuk  +  Gkwk’  k  =  0,  1 . N-l  (2.1) 
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where  x^eR“,  k  =  0, is  the  state  vector,  u^eRm,  k  =  0,  1,  .  . ,  N-l , 

is  the  control  vector,  w^eR1,  k  =  0,  1,  ... ,  N-l,  is  the  disturbance  vector, 

and  A^»  B^,  G^,  k  =  0,  1,  ,  N-l  are  given  matrices. 

It  is  assumed  that  the  initial  state  x  is  known  and  that  the  disturbance 

o 

vectors  w^  belong  to  given  nonempty  sets  W^C  Rr 

wk€Wk’  k  =  0,  1,  .  . . ,  N-l  (2.2) 


Attention  is  restricted  to  control  laws  of  the  form 


«n  _  m 
M-i, :  R  —  R  . 


k  =  0, 1,  . .  .,N-1 


(2.3) 


taking  values 


Uk  =  ^k(xk*’ 


k  =  0,  1,  ....  N-l 


(2.4) 


It  is  required  to  find  (if  it  exists)  the  control  law  in  this  class  for  which  the 
cost  functional 

N 

J^o’^r  •  •*>iN-l)  =  .S/fk(xk)  +  gk-l^k-l(xk-l)^ 

w.cW.  k=l 

k=0,  1 ,  . . ,  N-l  <2*5) 

is  minimized,  subject  to  the  system  equation  constraints  (2.  1),  and  where 
the  functions  f^R*1  ( -oo ,  +oo] ,  j :  Rm  -*  (-ao,  +oo] ,  k  =  1 , 2,  ....  N, 

are  given  closed  proper  convex  functions. 


In  Definition  A.  4  of  Appendix  I,  a  closed  proper  convex  function 
f :  Rn  (-oo,  +oo]  is  defined  to  be  an  extended  real  valued  convex  function 
which  is  lower  semicontinuous  and  such  that  -oo  <  f(x)  for  all  xeRn  and  with 
f(x)  <  +oo  for  at  least  one  xcRn.  Closed  proper  convex  functions  are  reviewed 
in  more  detail  in  Appendix  I.  One  of  the  advantages  of  using  extended  real 
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valued  functions  in  the  cost  functional  (2.  5)  is  that  state  constraints  and 
control  constraints  of  the  form  x^fX^,  u^eU^  where  X^,  are  given 
convex  sets  can  be  conveniently  incorporated  in  the  cost  functional  rather 
than  stated  explicitly.  This  is  accomplished  by  adding  under  the  summation 
sign  in  the  right  hand  side  of  equation  (2.  5)  the  indicator  functions 


4<*Jxk>  * 

«<uk|uk)  = 


if  xkcXk 

“  Vxk 

if  Uk6Uk 

ifVuk 


Since  the  theory  of  extended  real  valued  convex  functions  is  well  estab¬ 
lished,^^  introduction  of  the  extended  real  line  does  not  create  difficulties 
as  long  as  one  is  careful  to  avoid  the  meaningless  sums  oo  -  oo  and  -db  +oo. 

The  optimal  controller  in  Problem  2. 1  is  required  to  be  in  feedback 
form.  As  a  consequence,  local  variational  analysis  is  very  difficult  for  this 
problem  and  dynamic  programming  remains  the  only  method  to  proceed  for 
solution.  The  development  of  the  dynamic  programming  algorithm  for 
Problem  2.1  is  the  object  of  the  next  section. 


3.  Solution  by  Dynamic  Programming 


Let  us  denote  by  J  the  optimal  value  of  the  cost  functional  (2.  5) 

xo 


=  M'j »  •  •  •  >  Ppj  j)  (2.6) 

°  Hk 

k=0,  1,  .  .,N-1 


The  dynamic  programming  algorithm  to  be  described  in  the  following  pro¬ 
position  provides  the  optimal  value  Jx  at  the  last  step  of  a  recursive  se- 
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quence  of  minimization  and  maximization  steps.  Furthermore  the  optimal 
control  law  (if  it  exists)  can  be  obtained  from  the  sequence  of  the  minimi¬ 
zation  steps  in  a  much  simpler  way  than  directly  from  the  equation  (2.6). 


Proposition  2.1;  Assume  that  for  the  functions  defined  below  we  have 

-oo  <  H^(x^)  for  all  x^eRn  and  k  =  0,  1,  .  .  . ,  N-l.  Then  the  optimal  value 

J  of  the  cost  functional  (2.  5)  is  given  by 
xo 


J  =  J  (x  ) 

X  o  o 

o 


(2.7) 


where  the  function  Jq  :  R  —  (-oo,  +oo]  is  given  by  the  last  step  of  the  re¬ 
cursive  algorithm 


^N^XN^  =  *N^XN^ 

Ek+1(x)  =  8UP  Jk+l^X  +  GkWk*’  k  =  °»  1 . N_1  (2-9) 

^cWk 

Hk^Xk^  =  inf  ^Ek+l*Akxk  +  BkUk*  +  gk^Uk^’  k  =  °»  •  -  .N-!  (2.10) 

Uk 

Jk(xk)  =  Hk^Xk^  +  f{Jxk^’  k  =  1 . 2,  . . . ,  N-l  (2.11) 

J  (x  )  =  H  (x  )  (2.12) 

o'  o  o'  o 

Proof:  Since  -oo  <  j(xjsj_j)  for  all  x^  j€Rn,  we  have  that  for  every 
e  >  0  there  exists  a  function  :  Rn  —  Rm  such  that 


EN^AN-1xN-1  +BN-1,aN-1,  e^xN-l^  +  gN-  1^  N-  1 ,  e*xN-l  H 

-  inf  ^  EN^  AN-1xN-1  +BN-1^N-1^-1^  +  %U1^N-1*XN-1  ^  ^  +  c 
^N-l 


HN-1^xN-1^  +  6 


(2.13) 
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By  using  equations  (2.6)  and  (2.  9)  we  have 

7x  =  inf  .S,{£k(xk)  +  8k-l1,ik-l(xk-l,1) 

0  wkeWk  k=1 

k=0,l,..,N-l  k=0,l,..,N-l 

N-l 

=  inf  sup  S  ^k*xk*  +  gk-l^k-l(xk  1^ 

\  ^N-l  VWk  k=1 

k=0,l,..,N-2  k=0,l,..,N-2 


+  EN^  AN-1xN-1  +  BN-1h’N-1^xN-1^  +  gN-l^N-l(xN-l^ 

N-l 

i  M  "UP  2{fk<xk)  +gk.1(Vl<Xk-l)l) 

^k  VWk  k=1 

k=0,l,..,N-2  k=0,l,..,N-2 

+  EN^  AN-1xN-1  +  BN-l*AN-l,e  *XN-1)]  +  gN-ll  ^N-l.e  (xN-l)J 


Using  (2.13)  to  strenghthen  the  above  inequality  we  obtain 

J  <  inf  sup  2<f  (xk)  +  g  1lpk.,(xk  )]} 

0  ^k  ^cWk  k=1 

k=0, l,..,N-2  k=0,l,..,N-2 


+  Hn_1(xn_1)  +  e 

N-l 

sup  inf  S  {fk(xk*  +  gk-l^k  2  (xk- 1 

^k  VWk  ^N-l  k=1 

k=-0,l,..,N-2  k=0,l,..,N-2 


+  En[  an_ixN-1  +  BN-1^N-1^xN-1^  +  gN-l^N-l^xN-l^  +e 


(by  using  the  minimax  inequality) 


< 


N-l 

inf  Supw  ^£k^xk> +  ^k- ^k- l^xk-l ^ 

^N-l  wkfWk  k=1 


k=0,l,..,N-2  k=0,  l,..,N-2 
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+  enIAN-1xN-1  +  BN-1m,N-1<xN-1^  +  gN-l^N-l^xN-l^  +  c 


=  J  +  e 
x 

o 


Since  the  above  relations  hold  for  every  e  >  0  we  conclude  that 


inf 


f  N-l 

u 


8 “P  1  ,  {fk(xk>  +  8k- 1  K-lf*k-l®  +HN./,‘N-1)1 

V  Wk 


k=0,l,..,N-2  k=0,l,..,N-2 


By  repeating  the  above  procedure  we  eventually  obtain 


J  =  H  (x  )  =  J  (x  ) 
X  o  o  o  o 
o 


w.E.D. 


We  remark  that  the  value  of  the  function  l  la  point  s  the 

usual  interpretation  of  the  "cost-to-go"  from  the  point  x^  at  time  k.  This 
value  can  be  a  real  number  or  oo  but  by  the  assumption  -ao  <  H^fx^),  for  all 
x^cR11  it  cannot  be  -ao. 

The  occurance  of  the  equality  H^(x^)  =  -oo  for  some  x^eRn  indicates 
a  degeneracy  in  the  problem  statement  and  in  particular  in  the  cost  func¬ 
tional  chosen.  It  implies  the  existence  of  control  laws  which  result  in  a 
value  of  "cost-to-go"  which  is  arbitrarily  small  starting  at  state  x^  at  time 
k,  thus  indicating  that  the  optimization  problem  is  not  well  posed.  The 
assumption  -oo  <  H^(x^)  for  all  x^eRn  and  all  k  can  be  guaranteed  to  hold 
under  quite  general  assumptions  which  will  be  stated  in  the  next  section. 

The  occurance  of  a  value  H^(x^)  =  oo  for  some  x^eRn  has  an  interesting 
interpretation  in  the  case  where  the  constraint  sets  for  the  disturbance 
vectors  are  bounded.  It  should  be  recalled  that  the  extended  real  valued 
functions  f,  and  g,  in  the  cost  functional  (2.  5)  specify  constraint  sets  for 
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the  state  and  the  control  vectors.  It  must  be  x^eX^  and  u^eU^  for  all  k 
where  the  sets  and  are  given  for  all  k  by 

xk =  W1  W  <  °°} 

Uk  =  ^uklgk*uk*  <  00  ^ 

A  value  H^(x^)  =  oo  implies  that,  starting  from  the  state  x^  at  time  k,  for 
every  control  law  that  the  controller  uses'  subject  to  the  control  constraints, 
there  exist  disturbance  vectors  within  the  given  sets  which  will  cause  a 
violation  of  a  state  constraint  at  some  later  stage.  It  should  be  noted  that  if 

there  do  not  exist  any  state  constraints,  i.e.,  the  functions  f^  are  real  valued, 

l 

and  in  addition  the  sets  are  bounded  then  we  will  have  H^(x^)  <  oo  :for  all 

x,eRn  and  all  k. 
k 

The  value  of  the  optimal  control  law  at  a  point  can  be  obtained 
from  the  dynamic  programming  algorithm  as 

•VV  =  "k  <2- 14) 

where  u^  is  a  point  (assuming  it  exists)  where  the  infinum  in  equation  (2.  10) 
is  attained  for  the  fixed  point  x^.  In  the  case  where  for  the  fixed  point  x^ 
the  infimum  in  (2.10)  is  attained  at  more  than  one  point  the  equation  (2.  14) 
still  holds  with  u^  being  any  one  of  those  points. 

Aside  from  the  dimensionality  problem,  common  to  every  algorithm 
of  this  nature,  an  additional  drawback  of  the  dynamic  programming  algorithm 
is  the  maximization  indicated  in  equation  (2.9).  It  will  be  shown  later  that, 
under  some  quite  general  assumptions,  the  functions  are  convex.  There¬ 
fore  if  the  set  is  a  compact  polyhedron  the  search  for  the  supremum  in 
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equation  (2. 9)  can  be  confined  to  the  finite  set  of  the  vertices  ((Rl),  Th. 

32.2)  of  thus  partly  alleviating  the  computational  requirements.  In 

some  cases  however  the  sets  are  only  indirectly  known  via  their  support 

functions.  This  will  often  occur,  for  example  if  the  discrete  time  system 

(2.  1)  results  from  sampling  a  continuous  time  linear  system.'  '  In  this 

case  approximation  of  the  sets  by  a  polyhedron  is  possible  with  any 

desired  degree  of  accuracy.  If  however  this  approximation  is  considered 

(W2) 

undesirable, use  of  a  dual  algorithm  '  '  based  on  equations  which  will  be 

presented  in  the  next  section  may  be  advantageous. 

In  any  case  the  DP  algorithm  provides  a  good  starting  point  for  ob¬ 
taining  existence  results  and  necessary  conditions  for  optimality.  In  the 
following  section  its  properties  will  be  investigated.  In  particular  properties 
of  the  functions  F^,  H^,  J^,  k  =  0,  1,...,N-1,  of  equations  (2.  9)  through 
(2.  12)  will  be  deduced.  In  addition  the  question  of  ocistence  of  optimal  con¬ 
trol  laws  will  be  considered. 

4.  Properties  of  the  Dynamic  Programming  Algorithm 

and  Existence  of  Optimal  Control  Laws 

Properties  of  the  dynamic  programming  algorithm  will  be  investigated 
under  assumptions  which  cover  most  special  cases  of  the  general  Problem 
2.  1  which  are  of  practical  interest.  Under  these  assumptions,  the  question 
of  existence  of  an  optimal  control  law  will  be  answered  satisfactorily.  It 
should  be  noted  that  the  statement  "an  optimal  control  law  exists",  as  we  will 

use  it  here,  means  that  for  every  point  x^fRn  and  for  every  k,  k  =  0,  1 . N-l, 

there  exists  a  vector  u^eRn  ouch  that  the  infimum  in  equation  (2.10)  is  attained. 
This  does  not  exclude  the  possibility  that  this  infimum  is  oo.  With  this  inter- 
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pretation  if  for  some  x^eR11  we  have  E 

r, 

all  u^e  R  then,  from  equation  (2.  10),  H^(x^)  =  oo  and  the  infimum  in  (2. 10) 
is  attained  for  every  u^eRn#  This  in  turn  according  to  our  terminology 
implies  existence  of  an  optimal  control  law  in  as  much  as  the  point  x^  is 
concerned. 

The  point  of  view  that  we  adopt  concerning  the  existence  of  an  optimal 
control  law  coincides  with  the  usual  point  of  view  whenever  the  given  initial 
condition  xq  is  such  that  the  optimal  value  of  the  cost  functional  Jx  is  finite. 
As  explained  in  the  previous  section,  whenever  the  sets  are  bounded,  a 
value  J  =  oo  may  occur  due  to  the  presence  of  state  constraints  x,  eX,  im- 

XQ  K  K 

plied  by  the  functions  f^  in  (2.  5)  where 


ktiiVktBkV  +  *k,uk)  =  “’  £or 


xk  -  {*klfk(xk)<oo! 


There  may  also  exist  control  constraints  u^cU^  implied  by  the  functions  g^ 
in  (2 .  5) 

Uk  =  {ukl8k(uk>  <  001 

A  value  Jx  =  oo  indicates  that  for  every  control  law  lJLjc(xjc)»  k  =  0,  1,  .  . ,  N-l, 

there  exist  disturbance  vectors  w^eW^,  k=  0,1,.,.,  N-l,  which  will  cause 

either  a  violation  of  a  control  constraint  or  a  violation  of  a  state  constraint 

at  some  stage  during  the  operation  of  the  closed-loop  system.  In  other  words 

a  value  J  =  oo  indicates  that  there  does  not  exist  a  control  law  which  can 
xo 

guarantee  the  satisfaction  of  all  the  constraints  of  the  problem.  The  question 
of  the  existence  of  such  a  control  law  will  not  be  considered  in  this  chapter. 

This  question  however  is  central  in  the  problem  of  the  reachability  of  a 

target  tube  and  will  be  answered  in  the  context  of  that  problem  in  the  next  chapter. 
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In  order  to  avoid  some  rather  uninteresting  but  analytically  irri¬ 
tating  situations  we  will  make  the  following  assumption  which  will  hold 
throughout  the  remainder  of  this  chapter. 

Assumption  2.1: 

(a)  Each  of  the  functions  J^,  E^,  of  equations  (2. 8) 
through  (2.  12)  is  not  the  constant  +oo  function. 

(b)  The  sets  k=  0,1,...,  N-l,  are  compact. 

Before  we  proceed  with  stating  the  assumptions  under  which  we  will 
examine  the  problem  of  existence  of  an  optimal  control  law,  let  us  consider 
the  circumstances  under  which  the  minimum  of  a  convex  function  f  :Rn  -* 

(-oo,  oo]  may  not  be  attained.  If  f  is  lower  semicontinuous  the  only  such 
situation  arises  when  f  decreases  monotonically  along  some  direction  with  the 
result  that  either  the  function  is  not  bounded  below  or  the  infimum  of  the  func¬ 
tion  is  finite  but  "attained  at  infinity.  "  Typical  examples  are  the  functions 
f(x)  =  x  and  f(x)  =  e  with  xcR.  Thus  in  order  to  prove  the  existence  of  a  mi¬ 
nimizing  vector  it  is  necessary  to  impose  some  conditions  which  will  guar¬ 
antee  that  the  function  will  not  decrease  monotonically  (recede)  along  some 
direction.  Such  conditions  involve  the  notion  of  a  direction  of  recession  of  a 
closed  proper  convex  function.  This  notion  is  introduced  in  Definition  A.  10 
of  Appendix  I  and  its  importance  in  providing  existence  results  for  optimiza¬ 
tion  problems  is  stressed  in  Proposition  A.  23  of  Appendix  I.  The  assump¬ 
tions  concerning  the  cost  functional  (2.  5)  which  we  will  make  involve  this 
notion.  We  shall  consider  the  following  special  cases. 

Special  Case  R:  In  the  cost  functional  (2. 5)  every  direction  of  recession  of 
each  of  the  functions  f^,  k  =  1 , 2,  .  . ,  N,  and  g^,  k  =  0,  1 ,  . .  . ,  N-l ,  is  a  direc¬ 


tion  in  which  this  function  is  constant. 
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Notice  that  a  closed  proper  convex  function  f  with  no  direction  of 
recession  is  characterized  by  the  fact  that  its  nonempty  level  sets 

F  =  {x|f(x)  <  a}  ,  a:  real  number 

3l  " 

are  compact,  a  requirement  satisfied  by  the  functions  f^,  gk  of  many  cost 
functionals  of  the  form  {2.  5)  which  are  of  interest  in  practice.  However 
we  allow  the  functions  f^  and  gk  to  have  directions  in  which  they  are  con¬ 
stant  in  order  to  retain  the  possibility  to  weight  in  the  cost  functional  only 
certain  components  of  the  state  and  control  vectors.  The  case  fk(xk)  = 
xj^Qx^  where  Q  is  only  positive  semidefinite  symmetric  matrix  is  a  typical 
example  of  such  a  situation.  The  basic  property  of  a  function  belonging  to 
the  special  case  R  is  that  there  does  not  exist  any  halfline  {z|z  =  x+Xy,  X  >  0} 
originating  at  some  point  xcRn  and  pointing  in  the  direction  of  some  vector 
ycRn  along  which  the  function  is  monotonically  decreasing.  This  excludes 
the  possibility  that  either  the  value  fk(xk)  or  the  value  gk(uk)  decreases 
monotonically  as  either  x^  or  u^  become  arbitrarily  large  (in  norm)  along 
some  direction. 

A  second  special  case  which  we  will  consider  is  the  following: 

Special  Case  C:  The  functions  g^,  k  =  0,  1,  .  .  . ,  N-l,  of  the  cost  functional  (2.  5) 
have  a  recession  function  of  the  form 

(gk0+)(z)  =  +oo  for  z  4  0,  (gk0+)(0)  =  0,  k  =  0,  1 .....  N- 1 

The  notion  of  the  recession  function  of  a  closed  proper  convex  function 
is  introduced  in  Definition  A.  9  of  Appendix  I.  Essentially  the  condition 

(gkn+)(z)  4  +00  for  z  /  0,  (gk0+)(0)  =  0 


for  z  4  0, 
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requires  that  the  penalty  to  the  controller  for  using  control  vectors  large 
in  norm  is  sufficiently  great.  For  example  a  function  g^  does  not  satisfy 
this  condition  if  it  is  uniformly  Lipschitz  continuous.  On  the  other  hand 
the  requirement  of  the  special  case  C  is  satisfied  if  the  set  < 

=  u^Ru^  where  R  is  a  positive  definite 

symmetric  matrix. 

Throughout  this  chapter  we  shall  use  the  assumption: 

Assumption  2.2:  The  cost  functional  (2.  5)  satisfies  the  requirements  of 
either  Special  Case  R  or  Special  Case  C. 

We  are  ready  now  to  prove  the  following  proposition  which  states  that 
under  our  assumptions  convexity  and  lower  semicontinuity  are  preserved  in 
the  dynamic  programming  algorithm  and  that  optimal  control  laws  exist. 


oo)  is  compact  or  if  for  instance  g,  (u,  ) 


»sition  2.2: 


(a)  Under  the  Assumptions  2.  1  and  2. 2  the  functions  J^,  E^, 
of  equations  (2.  8)  through  (2.  12)  are  closed  proper  convex 


functions  for  all  k.  This  implies  in  particular  that  the  as¬ 
sumption  -oo  <  H^(x^),  for  all  x^eR"  in  Proposition  2.  1 
holds  for  all  k. 


(b)  The  supremum  in  equation  (2.9)  is  attained. 

(c)  An  optimal  control  law  exists. 


Proof:  Consider  first  the  function 


EnW  =  »up  fN(*  +  GN.1wN.1) 


wN-leWN-l 
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By  Proposition  A.  10  of  Appendix  I  and  using  also  Assumption  2.  1  the  func¬ 
tion  is  a  closed  proper  convex  function.  The  supremum  is  attained  for 
every  x  since  the  function  Jj^wN-l^  =  +  ^N-l WN-1  ^  is  ^°wer  semicon- 

tinuous  by  Proposition  A.  12  of  Appendix  I  and  the  set  j  is  compact. 


N-l 


(x)  =  fN(x  + 


Notice  also  that  for  all  w*.  ,cWXT  ,  the  function  of  x  E.. 

N-l  N-l  N,  w 

Gw  .  wNI  .)  has  the  same  recession  function  EM  0+  =  fM0+,  and  thus 

N-l  N-l'  N,  Wv  ,  N  * 

+  +  '  * * 

by  Proposition  A.  10  of  Appendix  1  EN°  =  fM°  •  Thus  if  every  direction  of 


N 


recession  of  f^  is  a  direction  in  which  it  is  constant  the  same  is  true  for  the 


function  EXI. 

N 


Consider  now  the  function  H 


N-l 


HN-l(xN-l)  =  inf  ^EN^AN-1xN-1  +  BN-1UN-1*  +  8n-1(uN-1^ 

UM  1 


N-l 


(2.14) 


By  equation  (A.  2)  of  Appendix  I  the  function  is  given  by 


HN-1  =  l  END(_BN-1,8n-i1AN-1 


(2.15) 


where  the  notation  in  the  above  equation  is  introduced  in  Propositions  A.  4 
and  A.  6  of  Appendix  I.  By  using  Proposition  A.  13  of  Appendix  I  we  have 
that  both  in  the  special  case  C  and  in  the  special  case  R  the  function  j 
is  a  closed  proper  convex  function  and  that  the  infimum  is  attained  in  equa¬ 
tion  (2.  14).  Also  in  the  special  case  R  every  direction  of  recession  of  the 
function  Hj^  j  is  a  direction  in  which  Hj^  ^  is  constant.  The  same  is  true 
for  the  function  ^  ^  +f^  j  since  by  Proposition  A.  9  we  have 

j0+  =  j0+  +  f^  j0+  and  every  direction  of  recession  of  the  function 

f^  j  is  a  direction  in  which  f^  ^  is  constant.  Thus  the  proposition  is  proved 
for  k  =  N  and  all  the  necessary  facts  have  been  established  so  that  we  can 
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proceed  in  exactly  the  same  manner  to  prove  the  proposition  for  k  =  N-l 

and  recursively  for  all  k.  Q.E.D. 

The  equation  (2.15)  shows  that  the  "cost-to-go"  function  ^  can 

be  obtained  from  the  functions  E^  and  ^  through  operations  that  have 

(R 1 ) 

been  extensively  studied  in  the  literature.  '  This  fact  is  very  helpful  in 
the  search  for  sufficient  conditions  for  existence  of  optimal  control  laws. 

For  example  stronger  sufficient  conditions  can  be  derived  in  the  case  where 
the  functions  and  g^  of  the  cost  functional  (2.  5)  are  polyhedral.  By  making 
use  of  the  results  of  Section  19  in  (Rl)  it  can  be  readily  proved  that  the  Pro¬ 
position  2.2  holds  for  this  case  under  assumptions  that  are  weaker  than 
Assumption  2.2. 

The  equation  (2.15)  can  be  used  also  for  calculating  the  conjugate 
functions  and  via  Propositions  A.  14  and  A.  15  of  Appendix  I.  We  have: 

H*(x*)  =  cUA^E^Jx*)  +  g*(-Bjx*)]}  (2.16) 

J*(x*)  =  cl{H£(x*)Of£(x*)}  (2.17) 


where  the  closure  operation  cl{*  }  and  the  infinal  convolution  operation  □ 

are  introduced  in  Definition  A.  5  and  Proposition  A.  4  of  Appendix  I.  The 
$ 

conjugate  E^+j  of  the  function  is  given  by 

e£+i(x*)  =  conv(j£+1(x*)  -  o-(Gjtx*|wk))  (2.18) 


where  y(*  |w^)  is  the  support  function  of  the  set  and  the  convex  hull 
operation  is  as  in  Definition  A.  3  of  Appendix  I.  The  equation  (2.18)  follows 
directly  from  Proposition  25  in  (W2). 
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The  equations  (2.  16),  (2.  17)  and  (2.  18)  can  form  the  basis  for  a 
dual  algorithm  for  calculating  the  optimal  cost  similar  to  the  one  proposed 
in  (W2).  The  implementation  of  this  algorithm  will  not  be  discussed  in 
this  thesis.  A  special  case  has  been  analyzed  in  detail  in  (W2). 


5.  Necessary  Conditions  for  Optimality 


In  dynamic  optimization  problems  necessary  conditions  for  optimality 

are  usi’*. '  'r  expressed  in  terms  of  the  costate  vector  and  the  related  adjoint 

(PI ) 

equation.  This  is  true  for  the  case  of  the  Pontryagin  Minimum  Principle' 
as  well  as  the  Minimax  Principle  of  Zero  Sum  Differential  Games  as 
described  by  Isaacs. In  both  these  cases  at  points  of  an  optimal  trajectory 
where  the  "cost-to-go"  function  is  differentiable,  the  costate  vector  is  equal 
to  the  gradient  of  the  "cost-to-go"  function.  In  light  of  this  fact  it  is  not 


surprising  that  the  necessary  conditions  for  optimality  which  we  derive  for 
the  minimax  problem  of  this  chapter  involve  vectors  in  thfe  subdifferentials 
(generalized  gradients)  of  the  "cost-to-go"  functions  J^,  of  the  equations 
(2.8)  through  (2.  12).  The  notion  of  the  subdifferential  &f(x)  of  a  convex 
function  of  at  a  point  x  is  introduced  in  Definition  A.  12  of  Appendix  I  and  some 
of  the  pertinent  facts  are  summarized  in  subsequent  propositions.  It  should 
be  noted  that  the  use  of  subdifferential  theory  in  the  analysis  is  necessitated 
by  the  fact  that  the  "cost-to-go"  functions  and  will  in  most  cases  be 
nondifferentiable  even  if  the  functions  f^  and  g^  in  the  cost  functional  (2.  5) 
are  real  valued  and  differentiable.  This  is  mainly  due  to  the  maximization 
indicated  in  equation  (2. 9)  as  will  be  shown  later. 

We  now  prove  the  following  necessary  conditions  in  order  that  the 


supremum  and  infimum  in  the  equations 


-31- 


Ek+l(X>  =  *up  Jk+l^x  +  Gkwk^’  k  =  0,  1....N-1 
w,  €W, 
k  k 


(2.9) 


Hk^xk^  =  inf  ^Ek+l*Akxk  +  Bkuk^  +  gk^uk^'  k  =  °»  *•  ' ' »  N_1 


(2.10) 


are  attained  at  given  point! . 


Proposition  2.3:  For  a  fixed  point  xeRn  let  w^eW^  be  a  point  where  the 

g|r 

Bupremum  is  attained  in  equation  (2.  9).  Then  for  all  vectors  xic+if c)Jk+1  (x+ 
Gk^k)  we  have 

<xk+P°kV  =  <xk+rGkwk> 

Wk  k 

/ 

where  Cx  +  denotes  the  aubdifferential  of  the  function  Jk+J  at  the 

point  (x  +  Gkwk). 

Proof:  Let  xk+l€  ^Jk+1(x  +  Gkwk).  By  Proposition  A.  18  of  Appendix  I  we  have 


Jk+l(x  +  CW  =  <xk+l,x  +  GkWk>  "  Jk+l(xk+l) 


(2.19) 


By  equation  (2.18)  we  have 


Ek+l(xk+l)  =  conv^Jk+l(xk+l)  -<r(Gkxk+i  l'Wk^ 


*  .  * 


!  JZ+lO-'(G!x*  .|W.) 


Using  the  above  inequality  in  equation  (2.  19) 

JWl<x  +  Glr^lr>  5  <x  +  Gkwk,x*+1>  -  Ek+1(xk+1)  -  o,(G(rx*^1 1 W^) 


'k~k+l'  "k' 


On  the  other  hand 


-32- 


<x-xk+i>  -  Ek+i(xk+i)  -  8*up  {<x'xk+i>  -  Ek+i(xk+i)} 


*  ,  * 


=  EuxiW  =  J1,X1  (x  +  Gi,wJ 


(2.21) 


Combining  the  inequalities  (2.20)  and  (2.21)  we  obtain 

J$C  jjc 

<Xlr4.1  »  -  ‘r(GkXk+J  Wk^  =  maX 


WkCWk 


kk+r^k"k 


which  proves  the  desired  equation.  Q.E.D. 


Consider  now  the  function  defined  by 
Hk(x)  =  inf  {Ek+1(*  +  Bkuk)  +  gk(uk)} 


(2.22) 


It  is  clear  that  if  for  a  fixed  point  x^eR  the  infimum  in  equation  (2. 10)  is  at¬ 
tained  at  a  point  then  the  infimum  in  equation  (2.22)  is  attained  at  the  same 
point  u^  when  x  =  A^x^.  Notice  that  for  all  x^eRn  we  have  H^(x^)  =  H(A^x^) 
and  that  =  Ek+j  0(-Bk)gk>  a  relation  which  is  proved  in  the  same  way  as 
equation  (A.  2)  in  Appendix  I.  From  Propositions  A.  14  and  A.  15  it  follows 
then  that  the  conjugate  convex  function  H*  of  the  function  is  given  by 


8,>  )  =  )  +  .v'_Bkx  1 


(2.23) 


We  now  have: 


Proposition  2.4:  For  a  fixed  point  x^eR  let  u^  be  a  point  where  the  infimum 
is  attained  in  equation  (2.  10).  Then  for  all  vectors  x*f^H^(A^x^)  we  have 


<x*,Bkuk>  +  gk(uk)  =  min  {<x*,  BkuR>  +  gk(uR)} 


where  is  the  function  defined  in  equation  (2.22). 

$  ^ 

Proof:  Let  x  e  3- H^A^x^) .  By  Proposition  A.  18  we  have 

Hk*Xk*  =  *MAkXk*  =  <AkXk’ X+>  ' 

or  by  equation  (2.23) 

»k<xk>  =  <Akxk'xS  -  Ek+l<x*>  '  8k'-Bk«*>  <2  24> 

On  the  other  hand  by  the  optimality  of  u^ 

Hk(V  =  Ek+l<Vk  +  Vk>  +  gk(5k> 

*  *JP  {<Akxk  +  Bk"k’x*>  -  Ek+l(x*>)  +  ®k^k> 

-  <Akxk* x*>  "  Ektl(x*’  +  <Bk“k'x*>  +  gk(“k)  (2-25) 

Combining  relations  (2.24)  and  (2.25) 

<x*  BkV+  *k("k>  i  -«k(-Bkx*' 

=  inf  {<x  ,  Bkuk>  +  gk(uR)} 

Uk 

which  proves  the  desired  equation.  Q.  E.  D. 

Notice  that  if  the  matrix  Ak  is  invertible  then  by  Proposition  A.  20 
of  the  Appendix 

d^k*Akxk*  =  A'k  ^Hk(xk) 

and  the  necessary  condition  of  Proposition  2.4  becomes 

<Ak  xk*Vk>  +  *k<"k> =  rnin{<Aklx^'Bkuk>+gk(uk»>Vxkf3Hk,xk> 
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A  sufficient  condition  for  the  infimum  to  be  attained  at  a  given  point 
in  equation  (2.10)  is  given  by  the  following  proposition: 


Proposition  2.  5:  Assume  that  for  some  vector  x^+j  and  some  vector  z  we 


have  (z)  and  that  for  a  vector  u^ 


<xk+l,BkV  +  gk^k)  =  min  '  Bvuir>  + 


kk+l  *  ~k~k  '  8k'“k' 


Then  we  have 


Hk,z  ■  Vk1  *  Ek+i<l) +  *k(5k> 


(2.26) 


i.e.,  the  infimum  in  equation  (2.22)  is  attained  at  the  point  u^  when  x  =  z  - 
B^u^.  In  addition  we  have  -  B^u^). 

Proof:  We  have  -  B^u^)  £  ®*k+l^  +  gk^k^  anc*  using  equations  (2.22) 

and  (2.23) 

V'-W  ’  Hk(l-Bk“k> 

-  <xk+l>l>  -  Ek+l(l)  +<-Bkxk+l'“k>  '  8k(ik> 


'  Ek+l<xk+l*  +  gk<"Bkxk+l)  1  ®k+l<xk+l’ 


=  .up  {<>£,,.  -  Bkuk>  -  Hk(.  -  Bkik)} 


Hence  equality  holds  in  the  above  algebra  implying  equation  (2.26)  and  that 


xk+l€SBk(z-Bkuk^  Q,E,D* 
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It  is  interesting  to  make  the  following  observation  in  Propositions 
2.3  and  2.4.  Consider  a  fixed  point  x^eRn,  and  let  be  a  vector  where 
the  infimum  is  attained  in  equation  (2.  10).  Let  also  w^  be  a  point  where 
the  supremum  is  attained  in  equation  (2.  9)  for  x  =  A^x^  +  B^u^.  Then  for 
any  vector  x  such  that 

x  OHk(Akxk)  f|  dJk+1(Akxk  +  Bkuk  +  Gkwk)  (2.27) 

we  have  from  Propositions  2.  3  and  2.4  that 
<x*,  BkGk  +  Gkwk>  +  gk(Gk) 

=  min  m»*  B.  u.  +  Gkwk>  +  g.  (u.  )}  (2.28) 

uk  wkeWk 

or  equivalently 

<x*.  Ajxk  +  BkGk  +  Gkwk>  +  gk(uk) 

*  min  max  {<x  .  Akxk  +  B,  uk  +  Gkwk>  +  Kk(u,  )) 
uk  «kcWk 

Notice  that  the  expression  within  bxaces  in  the  above  relation  is  the 
familiar  Hamiltonian.  It  is  evident  that  if  along  an  optimal  trajectory  one 
could  guarantee  for  every  k  the  existence  of  vectors  x  such  that  the  relation 
(2.27)  holds  and  find  a  law  for  propagation  of  these  vectors  (i.e. ,  an  adjoint 
equation)  then  the  Proposition  2.  3  and  2.4  would  be  pieced  together  into  a 
Minimax  Principle.  The  remainder  of  this  section  will  be  devoted  to  an 
effort  in  this  direction. 

We  first  give  the  definition  of  a  minimax  sequence  and  a  minimax 


trajectory: 
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Definition  2.1:  A  sequence  of  control  and  disturbance  vectors 
<VWo’  U1,W1 . uN-l,wN-l} 

is  called  a  mini  max  sequence  and  the  corresponding  trajectory  {xq,Xj . 

given  by 

*k+i  =  Vk +  Bk“k +  Gk*k  •  k  =  °> 1 . N- 1 

x  =  x 
o  o 

is  called  a  minimax  trajectory  if  for  all  k 

Hk(ik>  =  »k<Vk>  r  Ek+i(Ak;k  +  Vk>  +  Sk^k* 

*  inf {Ek+i<Vk +  Bkuk> + 

“k 

Ek+l(Akxk  +  ^“k1  =  Jk+l(xk+l’ 

=  ‘“P  Jk+l(Vk  +  Vk+Gkwk> 

VWk 

A  minimax  trajectory  results  during  operation  of  the  system  (2.1) 
when  an  optimal  control  law  is  used  and  when  disturbances  are  selected 
(by  Nature)  in  an  optimal  fashion.  It  is  evident  from  the  Definition  2. 1  and 
the  dynamic  programming  algorithm  of  Proposition  2.  1  that  if  a  minimax 
sequence  and  a  corresponding  minimax  trajectory  could  be  found  then  the 
optimal  cost  for  Problem  2.  1  would  be  obtained  as 

7*  =  W  =  .z,{fk(ik,  +  *k-i(Vi)) 

o  k=  1 


and  the  problem  would  be  at  least  partially  solved.  In  what  follows  we  obtain 
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a  necessary  condition  in  order  fora  control  and  disturbance  sequence  to 
be  a  minimax  sequence  under  some  special  assumptions. 

We  shall  make  the  following  assumption  concerning  the  convex  func¬ 
tions  H^,  defined  by  equations  (2.8)  through  (2. 12)  and  equation  (2.22) 

Assumption  2.3: 

(a)  For  all  k  the  range  of  the  matrix  contains  a 
vector  in  ri(dom  H^) 

(b)  For  all  k  we  have 

ri(domf^)  H  ri(domH^)  4 

where  the  relative  interior  of  the  effective  domain  ri(dom  • )  of  a  convex 
function  is  defined  in  Definition  A.  7  of  Appendix  I. 

The  assumption  (a)  above  is  needed  in  order  to  guarantee  by  Proposi¬ 
tion  A.  20  of  Appendix  I  that 

^Hk*Xk*  =  Ak9^k<Akxk) 

where  is  the  function  defined  in  equation  (2.22),  a  relation  essential  for 
the  proof  of  Proposition  2. 8.  This  assumption  will  hold  for  most  problems. 

In  particular  it  will  hold  if  the  matrix  A^  is  invertible  or  if  the  functions  f^ 
are  real  valued  in  which  case  it  can  be  easily  seen  that  the  functions 
will  also  be  real  valued  and  hence  domH^  =  ri(domH^)  =  Rn.  The  reason 
for  introducing  assumption  (b)  above  is  to  guarantee  by  Proposition  A.  19  of  the 
Appendix  I  that 

d<Jk(*k)  =  ^Hk^xk*  + 
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a  relation  also  essential  in  the  proof  of  Proposition  2.8.  This  assumption 
will  again  hold  for  most  problems  and  in  particular  it  will  hold  if  the  func¬ 
tions  f,  are  real  valued, 
k 

Assume  now  that  {uq,  wq,Uj,  Wj,  . . .  w^  j}  is  a  minimax  se¬ 

quence  and  {xQiXj,  ...,x^}  is  the  corresponding  minimax  trajectory.  The 
necessary  conditions  of  Propositions  2.3  and  2.4  hold  for  the  vectors  u^ 
and  w^.  Some  preliminary  facts  concerning  the  subdifferentials 


*Jk+l*xk+l 


)  will  be  proved  now  in  the  following  two  Propositions: 


Proposition  2.6:  For  all  k  =  0, 1,  .  . . ,  N-l  we  have 
^k<Akxk,C^Ek+l*Akxk  +  Bkuk) 

Proof:  Let  x^c^S^A^x^).  By  the  subgradient  inequality  (A.  4)  in  Appendix  I 
we  have 

Hk(s)  >  f$k(Akxk)  +<*  -  Akxk,x*>,  VzcRn  (2.29) 


Since  from  equation  (2.22) 

ak<Vk> 1  Ek+i<V:k +  Bk“k> +  *k(5k) 

Bk(l)  -  Ek+i(z  +  V  ' +  *k(ik> 

using  the  above  relations  in  (2.29)  we  obtain 

Ek+iu  +  Vk>  -  Ek+i(Ak;k  +  Vk>  +  <z  -  AkV  ***• *  z£R" 

+  _  _ 

which  implies  that  x  f&Ek+j(Akxk  +  Bkuk).  Q.E.  D. 


1 
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Proposition  2.7:  For  a  fixed  point  x  let  Wk(x)  be  the  set  of  points  where 
the  lupremum  is  attained  in  equation  (2.9).  Then 

dEk+1(x)  Dfconv{x*c5Jk+1(x  +  Gkwk)|  wke  Wk(x)}  (2.30) 

and  if  xe  int  (dom  Ek+j )  we  have 

^Ek+j(x)  =  conv{x*e5Jk+1(x  +  Gkwk)|wk€  Wk(x)}  (2.31) 

where  conv{*}  denotes  convex  hull  of  the  set  within  braces. 

Proof:  Let  wkcWk(x)  and  x*edJk+1(*  +Gkwk)  then 

Jk+,(«+Gkwk)  >  Jk+1(x  +  Gfcwk)  +<*-*, x*>,  Vmr"  (2.32) 

Since  Jk+,(x  +  Gkwk)  =  Efc+1(x)  and  J  k+1(*  +  Gkwk)  <  Ek+1(a)  from  the  relation 
(2.32)  we  obtain 

Ek+1(s)  1  Ek+lM  +<z'x'**>'  VaeRn 
$ 

implying  that  x  e^Ek+j(x)  and  therefore 

5Ek+l (x)  =  S Jk+1  (x  +  Gkwk)  .  V  »k'Wk,x) 

Since  dEk+j(x)  is  a  closed  convex  set 

^Ek+1  D  conv{x%aJk+1  (x  +  Gkwk)  |  wkf?k(x)) 

To  prove  the  equality  (2.31)  observe  that  the  function^  (x,  wk)  =  J^+j(x  + 

Gkwk)  satisfies  all  the  assumptions  of  Proposition  A.  22  of  Appendix  I.  The 
equality  (2.31)  follows  directly  from  the  conclusion  of  this  proposition.  Q.E.D. 
From  Propositions  2.6  and  2.7  it  cannot  be  guaranteed  that  the  set 


intersection  indicated  in  relation  (2.27)  is  a  nonempty  set,  and  in  fact 
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example  a  can  be  found  where 

*R(Akxk)0  SJk+l(AkXk  +  BkUk  +  Gkwk*  =<^ 

If  however  the  equality 

&Ek+l*Akxk  +  BkV  =  SJk+l*Akxk  +  Bkuk  +  Gkwk)  (2.32) 


holds,  then  from  Proposition  2.6  we  obtain 

dH(Akxk)  C  S  Jk+i(Akxk  +  Bkuk  +  Gkwk) 

in  which  case,  assuming  i  4>  ,  the  minimax  condition  of  equation 

(2.28)  would  hold  for  every  x  e5H^(A^x^). 

By  Proposition  2.7  the  equation  (2.32)  is  satisfied  for  every  point 
x  =  (Akxk  +  Bkuk)cint(dom  >W  for  which  the  supremum  in  equation  (2.  9) 
is  attained  at  a  single  point.  It  may  be  satisfied  also  for  other  points  on 
the  boundary  of  dom  Ek+j .  Points  (Akxk  +  Bkuk)  for  which  equation  (2.  32) 
is  satisfied  will  be  called  nonsingular  according  to  the  following  definition. 


Definition  2. 1 :  For  fixed  k,  k  =  1, 2,  . . ,  N,  a  point  x  is  called  nonsingular 
if  for  every  vector  wk  such  that  the  supremum  in  equation  (2.  9)  in  attained, 
we  have 

aEk+i,x)  ■  3Jk+i(x  +  Gk*k| 

A  point  x  which  is  not  nonsingular  will  be  called  singular. 


It  should  be  noted  that  in  view  of  Proposition  2.7,  every  point  x  at 
which  Ek+j  is  differentiable  is  by  necessity  nonsingular,  assumming  that 

^Jk+l(x  +  Gk;:'k),'*• 
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We  shall  also  need  to  distinguish  initial  states  xq  for  which  the 
subdifferential  dJQ(xo)  is  nonempty  by  the  following  definition. 

Definition  2.2:  The  initial  state  xq  will  be  called  regular  if  the  subdifferential 
is  nonempty. 

Notice  that  by  Proposition  A.  17  of  the  Appendix  all  initial  states 
xq€  ri(dom  Jq)  are  regular  whereas  all  initial  states  for  which  J0(xQ)  =  oo 
are  not  regular. 

We  are  now  ready  to  prove  the  following  necessary  condition  for  a 
minimax  sequence. 


Proposition  2.8:  Let  {uq,  wq,  u^,  Wj,  .  . . ,  u^_  j,  ^N-l  ^  be  a  minirnax  sequence 
and  let  {xo,Xj,  .  . . ,  x^}  be  the  corresponding  minimax  trajectory.  Assume 
that  the  initial  state  x  is  regular  and  that  the  points  (A^x^  +  Bkuk),  k  =  0,  1,  .  . , 

)jc  $  >J<  5{<  # 

N-l  are  nonsingular.  Then  there  exist  vectors  Xj ,  x^,  ...»  xN,  pj ,  p^,  ....  pN  j 
satisfying  the  adjoint  equation 

xk  =  AkXk+l  +  Pk  '  k=  1 . N_1  (2.33) 


with 

xNc^fN^xN^ 

Pk  ^S^k(xk)i  k  =  1,2 . N-l 


and  such  that 


<xk+l’  Vk  +  Gkwk>  +  8k("k) 

=  min  max  (<x^+1»  Bkuk  +  Gkwk>  +  gk(uk)},  k  =  0,1, 


uk  wkeWk 


.N-l 

(2.34) 
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Proof:  By  the  fact  that  x  is  a  regular  point  we  have  (x  )  4  b .  Since 
*■  o  ‘  o  o 

by  the  Assumption  2.3  we  have  dJQ(xo)  =  Ho(AoXq)  we  conclude  that 
(A  x  )  4  <b .  Take  x.  to  be  any  vector  in  gH  (Ax).  By  Proposition 

o  O  O  X  #  q  0  q  *  r 

2.  5  and  the  fact  that  the  point  (Ax  +  B  u  )  is  nonsingular  we  have 

o  o  o  o 

JVVo'c^i'Vo*  Vo1  = 

and  therefore  by  Propositions  2. 3  and  2.4  the  minimax  condition  of  equation 
(2. 34)  is  satisfied  for  k  =  0.  By  the  Assumption  2.  3  we  ihave 

dJjfxj)  =  A'jd  HjfAjXj)  +  S 
)|(  )jc 

and  thus  we  can  find  vectors  x2  and  such  that 

*  *  $ 

X1  =  A'ix2  +  P2 

and  x2eS  H^AjXj),  p2cSfj(*1)*  Again  by  Proposition  2. 5  and  the  fact  that 
the  point  (AjXj  +  BjUj)  is  nonsingular  we  have 

SHjCAjXjJcdEjlA^  +  BoGo)  =  SJ2(x2) 

and  therefore  by  Propositions  2.  3  and  2.4  the  minimax  condition  of  equation 

(2.34)  is  satisfied  for  k  =  1.  By  proceeding  in  a  similar  manner  we  construct 
*  *  $  $  $  $ 

the  sequence  x^ ,  x2>  .  .  . ,  x^,  P|  •  P2>  •  •  •  *  Pjvj  |  •  For  these  vectors  the  adjoint 
equation  (2.33)  as  well  as  the  minimax  condition  (2.34)  is  satisfied.  Q.E.D. 

The  Proposition  2.8  states  that  a  minimax  principle  holds  along  a 
minimax  trajectory  provided  this  trajectory  does  not  go  through  singular 
points  and  the  initial  condition  is  regular.  This  is  reminiscent  of  the  mini¬ 
max  principle  of  differential  games  which  holds  provided  the  optimal  trajec¬ 
tory  does  not  go  through  singular  surfaces. 
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Except  for  the  assumption  concerning  nonsingular  points  every 
other  assumption  used  in  the  proof  of  Proposition  2.  8  is  required  in  order 
to  rule  out  rather  pathological  cases  which  seldom  occur  in  practice.  Hov' 
ever  the  assumption  that  the  trajectory  does  not  go  through  singular  points 
is  a  formidable  one.  Except  for  particulady  well  behaved  problems, 
singular  points  are  a  common  occurance  and  invariably  minimax  trajec¬ 
tories  corresponding  to  some  initial  states  will  go  through  these  points. 
One  can  prove  in  fact  that  if  xQ  is  an  initial  state  which  is  such  that  there 
exists  a  minimax  trajectory  starting  from  xq  which  does  not  go  through 
singular  points  then  we  must  have 

JL<V  *  W  *  7x  <2-35> 


where  7*  =  JQ(X0)  i*  the  optimal  cost  of  Problem  2. 1  corresponding  to  xq 
o 

and  JL(*0)  i»  given  by 


jl(xo'  =  “jL  ‘n<  *,{W  +  *k.il,ik-i(xk-i)1} 

"k  K- i 

k=0,l,..,Nq  k=0,l,..,N-l 


The  equation  (2.35)  implies  the  existence  of  a  saddle  point  in  the  zero-sum 
game  where  the  players  are  the  controller  and  nrture  and  the  payoff  function 

is 

N 

.Vw +gk-iI,ik-i(xk-i)|> 

k=l 


Since  we  have  JT  (x  )  <  J  (x  )  with  strict  inequality  holding  in  general  for 

L  O  —  O  O 

a  "large"  set  of  initial  states,  the  equation  (2.35)  illustrates  the  limitations 
of  Proposition  2.  8. 
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J.t  appears  that,  in  general,  it  is  a  formidable  task  to  determine  the 
set  of  initial  states  which  are  such  that  their  corresponding  minimax  trajec¬ 
tories  do  not  go  through  singular  points.  The  same  is  true  for  determining 
whether  a  particular  point  is  singular  or  not.  Thus  even  if  a  candidate  for 
a  minimax  sequence  is  found  through  Proposition  2.  8  it  may  be  very  dif¬ 
ficult  to  verify  whether  in  fact  it  is  a  minimax  sequence. 

In  conclusion  the  necessary  conditions  prewented  in  this  section  should 
be  expected  to  provide  a  complete  solution  to  only  a  limited  class  of  problems. 
The  class  of  problems  for  which  the  singular  points  either  do  not  exist  at  all 
or  can  be  detected  by  graphical  or  analytical  methods. 

One  class  of  problems  where  singular  points  do  not  occur  is  the  case 
where  the  functions  f^  of  the  cost  functional  (2.  5)  are  linear 

fk(*k)  =  <xk.  ck>  ,  k  =  1,2,  ...,N-1 

where  <*,  •>  denotes  inner  product  and  ck  are  given  vectors  in  Rn.  If  the 
functions  gk,  k  =  0,  1,  .  . ,  N-l  satisfy  the  requirements  of  special  case  C 
then  it  can  be  easily  proved  that  the  functions  Ek>  Hk,  Jk  of  the  DP  algorithm 
are  linear  functions.  In  particular  the  functions  Ek  are  differentiable,  and 
therefore  singular  points  do  not  appear.  A  minimax  sequence  for  this  prob¬ 
lem  can  be  obtained  by  making  use  of  the  minimax  condition  of  Proposition 
2.  8. 

The  minimax  principle  of  Proposition  2.  8  however  can  be  used  in  still 
a  different  way.  Assume  that  a  sequence  {uq,  w^,  Uj  ,  Wj u^  j.w^,  j)  with 
a  corresponding  trajectory  {xq,Xj,  . .  .  ,  x^}  has  been  found  via  the  minimax 
condition  (2.34),  and  that  one  cannot  verify  whether  indeed  this  sequence  is 
minimax  sequence.  Let 
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7x  =  .Z,lfk(ikl  +8k-l<“k-l>l 
o  k=l 


be  the  value  of  the  cost  corresponding  to  the  sequence.  Then  one  can  easily 
prove  by  making  use  of  Propositions  2.3,  2.4,  and  2.  5  that  the  inequality 

Jx  <  Jx  (2.36) 


holds,  where  J  is  the  optimal  value  of  the  cost  functional  (2.  5).  In  some 
o 

cases  now  minimax  problems  are  solved  in  order  to  determine  the  optimal 

value  J  and  compare  it  with  the  worst-case  performance,  say  J  ,  of  a 
o  xo 

controller  selected  on  the  basis  of  other  considerations.  The  reasoning 

used  is  that  if  the  difference  (J  -  J  )  is  relatively  small  then  it  can  be 

o  o 

concluded  that  the  worst-case  performance  of  this  suboptimal  controller  is 
not  unduly  poor.  Since,  by  using  the  relation  (2.  36),  we  have 


°SJX  -Jx  iJxD 
o  o  o 


a  "small11  value  of  (J  -  J  )  can  guarantee  that  the  worst-case  perform- 

o  o 

ance  of  the  controller  under  consideration  is  acceptable. 


6.  Discussion  and  Sources 

The  basic  approach  towards  the  solution  of  the  problem  of  this  chapter 
is  dynamic  programming.  The  computational  requirements  for  this  algorithm 
depend  on  the  dimension  of  the  system  and  the  nature  of  the  sets  in  which 
the  disturbance  is  known  to  belong.  If  the  sets  are  compact  polyhedra 
with  a  relatively  small  number  of  vertices  the  computational  requirements 
are  only  slightly  greater  than  those  for  a  deterministic  optimal  control  prob¬ 
lem  with  the  same  state  and  control  vector  dimensions. 
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In  some  cases  a  dual  algorithm  involving  the  conjugate  convex  func¬ 
tions  of  the  "cost-to-go"  functions  can  offer  computational  advantages, 
particularly  if  the  sets  are  only  indirectly  known  via  their  support 
functions. 

New  results  in  this  chapter  are  the  existence  results  of  Proposition 
2.2  and  some  of  the  necessary  conditions  in  Section  4.  The  minimax  prin¬ 
ciple  of  Proposition  2.  8  should  not  be  considered  as  a  powerful  tool  for 
solving  a  wide  variety  of  problems.  It  can  be  useful  however  in  some 
cases  and  it  is  of  theoretical  interest  since,  together  with  the  developments 
preliminary  to  its  proof,  it  provides  insight  into  the  mechanism  of  optimality 
for  the  problem  of  this  chapter. 

Two  special  cases  of  Problem  2.  1  are  of  interest  in  deterministic  opti¬ 
mal  control  theory.  In  the  first  case  the  sets  consist  of  a  single  point  wk* 

=  {w^}  ,  k  =  0,  1,  .  .  .,N-1.  For  this  optimal  control  problem  the  Pro¬ 
position  2.2  yields  existence  results  that  to  the  author's  knowledge,  are 
stronger  than  those  available  in  the  literature.  Some  of  the  results  on 
existence  of  optimal  controls  in  Lee  and  Markus^^  are  along  the  same 
lines.  For  the  same  case  the  Proposition  2.8  yields  a  Minimum  Principle 
which  holds  for  a  linear  discrete-time  system  and  a  convex  but  not  differ¬ 
entiable  cost  functional.  This  Minimum  Principle  is  a  sufficient  as  well 
as  necessary  condition  for  optimality  as  can  be  easily  verified  by  using 
Proposition  2. 5.  Notice  that  for  this  case  there  exist  no  singular  points 
due  to  the  nature  of  the  sets  W^.  A  similar  Minimum  Principle  for  a  linear 
continuous -time  system  and  a  convex  but  not  differentiable  cost  functional 
has  already  appeared  in  (Hel).  Necessary  conditions  along  similar  lines 
can  also  be  found  in  (R2),  (Lu2),  (B3).  A  second  special  case  of  interest  in 
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deterministic  optimal  control  theory  is  the  case  where  the  system  is  de¬ 
scribed  by  the  equation 

=  A^Xj^  +  G^w^,  k=  0,  1,  . .  . ,  N-l  (2.36) 

and  it  is  required  to  find 

N 

J  =  sup  2  f  (x,  )  (2.37) 

o  wkeWk  k=l  K  K 

k=0,l,..,N-l 

where  fk»  k  =  1,2 . N  are  real  valued  convex  functions.  This  problem 

can  be  recognized  as  the  special  case  of  Problem  2.  1  with  the  functions 
gk  defined  as 

gk(uk)  =  00  for  uk  t  0.  gk(0)  =0,  k  =  0,  1 . N-l 


For  this  problem  Proposition  2. 8  yields  a  Maximum  Principle  which  can  be 
proved  without  the  assumption  that  the  optimal  trajectory  does  not  go  through 
singular  points,  and  that  the  initial  state  is  a  regular  point.  The  proof  is 
based  on  Propositions  2.3  and  2.7  and  an  argument  similar  to  the  one  used 
for  the  proof  of  Proposition  2.  8.  This  Maximum  Principle  holds  for  a  linear 
system  and  a  nondifferentiable  convex  functional  and  provides  a  necessary, 
but  not  sufficient,  condition  for  optimality  for  the  problem  of  equations  (2. 36) 
and  (2.  37).  It  can  be  easily  generalized  for  the  case  of  system  (2.  36)  where 


it  is  required  to  find 


Jx_  =  ■“?„  ,r1{W+hk.l‘-k.l» 


Y»k  k=1 
k=  0, 1,..,N-1 


where  h  k  =  0,  1,  ... ,  N-l,  are  any  continuous  real  valued  functions  on  Rr, 
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Open- loop  discrete  time  minimax  control  problems  can  be  viewed 
as  single-stage  feedback  problems  (N  =  1),  and  therefore  for  the  case  of 
a  linear  system  and  a  convex  cost  functional  they  can  be  considered  as  a 
special  case  of  Problem  2.1.  In  addition  to  the  results  of  this  chapter 
the  necessary  conditions  in  (Dal),  (Da2),  (Bral),  (Dl),  (D2),  (D3)  and  the 
computational  algorithms  in  (Dl),  (Sal),  (Psl),  (B3)  can  be  used  for  the 
solution  of  such  problems.  Some  of  the  material  in  these  references  is 
applicable  to  nonlinear  and  nonconvex  open-loop  problems  as  well. 

Linear  discrete -time  minimax  control  problems  with  perfect  state 

fWl )  (W2) 

information  were  considered  first  by  Witsenhausen'  ”  '  '  who  developed 

the  dynamic  programming  algorithm  and  its  dual  for  the  case  of  the  cost 
functional  J(fiQ,  Pj,  ...»  pN_j)  =  sup  f^(xN).  He  considered  in  detail 

wk^Wjt 

k=0,l,..,N-l 

the  implementation  of  the  dual  algorithm  for  thi3  case  and  gave  a  necessary 
condition  which  parallels  Proposition  2.4  of  this  chapter.  He  also  observed 
that  a  minimax  principle  in  general  does  not  hold  due  to  the  presence  of 
singular  points. 

The  dynamic  programming  algorithm  of  this  chapter  can  be  extended 
to  much  more  general  minimax  control  problems  as  will  be  seen  in  Chapter  5. 
All  the  other  results  of  this  chapter  rely  on  linearity  and  convexity.  Their 
extension  however  to  continuous -time  linear  systems  appears  to  involve 
great  technical  difficulties. 


CHAPTER  3 


REACHABILITY  OF  A  TARGET  TUBE  WITH 
PERFECT  STATE  INFORMATION 

1 .  General  Remarks 

In  this  chapter  we  consider  a  special  case  of  the  problem  of  the 
previous  chapter  which  will  be  referred  to  as  the  problem  of  the  Reach¬ 
ability  of  a  Target  Tube  by  the  state  of  the  system  when  the  controller 
has  available  at  each  time  a  perfect  measurement  of  the  system  state. 
The  motivation  for  considering  this  problem  arises  from  two  basic  prob¬ 
lems  of  deterministic  control  theory,  the  controllability  problem,  and 
the  tracking  (servomechanism)  problem.  The  controllability  problem 
is  concerned  with  transfering  the  state  of  a  system  from  an  initial  state¬ 
time  pair  to  a  final  state-time  pair.  The  tracking  problem  is  concerned 
with  keeping  the  state  trajectory  of  the  system  "sufficiently  close"  to  a 
prescribed  trajectory. 

The  problems  considered  in  this  chapter  can  be  viewed  as  the 
analogs  of  these  two  problems  when  there  are  disturbances  driving  the 
system.  In  accordance  with  the  general  approach  of  this  thesis  we  as¬ 
sume  that  these  disturbances  are  unknown  except  for  the  fact  that  they 
belong  to  given  sets.  Under  these  circumstances,  the  most  natural 
analog  of  the  deterministic  controllability  problem  is  that  of  steering 
the  system  state  at  the  final  time  into  a  desired  target  set  under  all 
possible  combinations  of  disturbances.  In  other  words,  we  would  like  to 
design  a  feedback  controller  in  such  a  way  as  to  guarantee  that  the  final 
state  of  the  system  will  always  lie  in  a  prescribed  target  set  despite  the 
presence  of  uncertainties.  In  a  similar  vein,  a  natural  analog  of  the 
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tracking  problem  under  these  same  conditions  is  to  keep  the  entire  state 
trajectory  in  a  "tube"  containing  the  desired  trajectory  under  all  possible 
disturbances.  We  refer  to  these  two  problems  as  those  of  "Reachability 
of  a  Target  Set"  and  "Reachability  of  a  Target  Tube".  Possible  applica¬ 
tions  of  these  two  problems  can  be  expected  in  the  control  of  systems 
under  uncertainty  when  either  a  set-membership  description  of  the  un¬ 
certain  quantities  is  more  readily  available  than  a  probabilistic  one,  or 
where  specified  tolerances  must  be  met  with  certainty. 

In  the  next  section  we  formulate  the  problem  of  Reachability  of 
a  Target  Tube  which  involves  a  linear  discrete-time  system.  The  prob¬ 
lem  of  Reachability  of  a  Target  Set  under  the  same  circumstances  can  be 
viewed  as  a  special  case  of  the  problem  of  Reachability  of  a  Target  Tube 
and  will  not  be  considered  explicitly.  The  solution  of  the  problem  by  dy¬ 
namic  programming  will  be  given  in  Section  3  by  making  direct  use  of 
the  results  obtained  in  Chapter  2.  In  Sections  4  and  5  we  shift  the  em¬ 
phasis  to  the  development  of  algorithms  which  have  potential  for  practical 
applications.  We  consider  the  case  where  all  the  given  sets  are  ellipsoids 
in  the  appropriate  Euclidean  spaces  and  we  develop  ar.  ellipsoidal  approxi¬ 
mation  algorithm  which  results  in  a  control  law  which  is  a  linear  function 
of  the  system  state,  thus  offering  attractive  implementation  advantages. 


Problem  Formulation 


We  will  consider  the  following  problem: 

Problem  3.1:  Given  is  the  linear  discrete-time  dynamic  system: 


xk+l  =  Akxk  +  BkUk  +  Gkwk  ’  k  =  °»  !*  *  *  *  • 


N-l 


(3.1) 
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where  x^c  Rn,  k=  0,  1, .  . ,  N,  are  the  state  vectors,  u^e  Rm,  k=0,  1,  .  . ,  N-l, 
are  the  control  vectors,  w^e  Rr,  k=  0,  ..,N-1,  are  disturbance  vectors,  and 
A^,  B^,  G^,  k=  0,  1,  .  ...  N-l  are  given  matrices  of  appropriate  dimension. 

The  initial  state  xq  is  known  and  the  disturbance  vectors  belong  to 
given  compact  sets  W^CR  ,  w^e  W^,  k  =  0,1,...,  N-l. 

Attention  is  restricted  to  control  laws  of  the  form 
Pk  :Rn  —  Uk,  k  =  0, 1,  . . .  ,N-1 
taking  values 

Uk  =  ^k^Xk^*  k  =  0,  1,...,N-1 

where  UkCRm,  k  =  0,  1,  .  .  ,N-1,  are  given  closed  convex  sets.  It  is  required 
to  find  (if  it  exists)  a  control  law  in  this  class  such  that  for  all  k  the  state 
xk+j  of  the  closed-loop  system 

xk+l  =  Akxk  +  BkJik(xk)  +  Gkwk  (3,2) 


is  contained  in  given  closed  convex  sets  Xk+j, 

sible  values  of  the  disturbance  vectors  w,  . 

k 


k  =  0,1, 


N- 1 ,  for  all  pos- 


We  shall  say  that  the  target  tube  {Xj,  X .  .  . ,  X^}  is  reachable  if 
there  exists  such  a  control  law. 

It  is  easy  to  see  that  the  Problem  3.  1  is  a  special  case  of  the  Prob¬ 
lem  2,1  of  the  previous  chapter  with  the  cost  functional  defined  as 


J<>w 


sup 

wkcWk 


k=  0,  1,  . 


N 

2  i«(xk|Xk)  +  6[^_1(xk_1)|Uk_1]}  (3.3) 
k  =  1 

N-l 


where  6(x|X)  denotes  the  indicator  function  of  the  set  X  ^6(x|X)  =  0  if 
xeX,  6(x|X)  =  ao  if  X  ^ 

With  this  definition  the  target  tube  {Xj,  X2,...,X^}  is  reachable  if  the  optimal 
value  J  of  the  cost  functional  (3.3)  is  0.  It  is  not  reachable  if  J  = 


00  . 
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It  should  be  noted  that  the  problem  of  this  section  has  also  been 
considered  in  a  somewhat  more  general  form  in(Bl).  The  approach  used 
in  this  reference  is  purely  geometrical  and  does  not  rely  on  the  solution 
of  the  Problem  2.1. 

3.  The  Dynamic  Programming  Algorithm 

Application  of  the  dynamic  programming  algorithm  cf  Proposition  2.1 
of  the  previous  chapter  yields  the  optimal  cost 

\  *  W  *  a<*Jxo>  <3-4> 

from  the  recursive  equations 

Ek+1(x)  =  6(x|Tk+1),  k=  0,2 . N-l 

Jk(xk)  =  6(xk|X*),  k=  0,1 . N 

where  the  sets  Tk  and  Xk  are  given  by  the  relations 


X 

* 

N  = 

XN 

(3.5) 

Tk+1 

=  (x 

|(x  +  GkWk)CXk+1},  k  =  0,1,  ...,N-1 

(3.6) 

x*  = 

Ak 

{xki 

*Akxk  +  Bkuk*eTk+r  £or  SOme  uk€Uk^nXk’ 

k=  1,2,  .  .,N-1 

(3.7) 

X*  = 
o 

(A  x  +B,u,  )eT.,  for  some  u  eU  } 

'  o  o  k  k'  1  o  o 

(3.8) 

$ 

If  xq€  XqI  by  equation  (3.4),  the  optimal  cost  is  0  implying  the  exis¬ 
tence  of  a  control  law  that  achieves  reachability  of  the  target  tube  {Xj,  X^,.., 

Xj^j}.  If  xq^Xq  then  the  target  nbe  is  not  reachable. 

$ 

Some  of  the  properties  of  the  sets  l'k  and  Xk  of  equations  (3.  5)  through 
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(3.8)  can  be  obtained  by  making  use  of  propositions  derived  earlier  in 
Chapter  2.  Thus  by  Proposition  2.2  these  sets  are  convex  whenever  nonempty,  and 


if  the  sets  are  compact  they  are  also  closed. 


If. in  laddi'tion  the  matrices  k  = 


0,  1,  . . .  ,N-1,  are  invertible  then  it  can  be  proved  that  the  sets  and 
$ 

are  compact.  Also  since  the  support  function  and  the  indicator  func¬ 


tion  of  a  closed  convex  set  are  conjugate  to  each  other  the  support  func 


tions  of  the  sets  and  can  be  obtained  by  making  use  of  the  equations 


(2. 16),  (2.  17)  and  (2. 18)  of  the  previous  chapter. 


A  control  law  that  achieves  reachability  can  be  obtained  as  follows. 
$ 

To  every  x^e  X^  associate  a  vector  |a.^(x^)  =  u^e  such'  that 


<Akxk  +  Bkuk,e  Tk+i 
* 

By  definition  of  the  set  X^  such  a  vector  exists.  It  can  be  seen  that  if  the 
target  tube  {Xj,  X ....  X^}  is  reachable  from  the  initial  state  of  the  sys¬ 
tem,  then  if  we  use  a  control  law  defined  as  above  the  state  x^  will  belong 
* 

to  the  set  X^  for  all  k,and  thus  definition  of  the  control  law  for  vectors 

outside  the  set  X.  is  redundant, 
k 

For  purposes  of  future  reference  the  tube  {Tj,  .  .  .  ,  T^}  will  be 

called  effective  target  tube.  The  tube  {X*,  X*,  . .  . ,  X^}  will  be  called 

modified  target  tube  and  in  fact  it  specifies  the  region  of  state  space  where 

the  state  will  lie  when  a  control  law  that  achieves  reachability  is  used. 

For  practical  applications  it  is  important  that  the  sets  T^  and  X^ 

of  the  effective  and  modified  target  tube  can  be  characterized  by  a  finite 

set  of  numbers.  This  is  possible  when  the  given  sets  X^  and  are  convex 

$ 

polyhedra.  The  sets  T^  and  X^  are  in  this  case  polyhedra  and  thus  can  be 
characterized  by  a  finite  number  of  bounding  hyperplanes.  The  corres- 
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ponding  algorithm'  '  is  however  beset  by  the  fact  that  the  number  of 
bounding  hyperplanes  increases  at  every  step  of  the  algorithm.  In  ad¬ 
dition  the  implementation  of  a  control  law  that  achieves  reachability  can 
be  quite  cumbersome. 

In  the  case  where  the  given  sets  are  not  polyhedra,  characteriza- 
$ 

tion  of  the  sets  T^»  X^  of  the  effective  and  modified  target  tubes  by  a 

finite  set  of  numbers  is  in  general  infeasible.  One  can  however  conceive 

of  constructing  sets  that  internally  approximate  the  sets  T^,  X^  and  which 

can  be  characterized  by  a  finite  set  of  numbers.  One  such  possibility  is 

$  — 

to  approximate  the  sets  and  X^  for  each  k  by  ellipsoids  T^CT^, 

sft  jjc 

X,,CX,_  since  an  ellipsoid  is  completely  characterized  by  its  center  and 

a  weighting  matrix.  Then  in  order  for  the  original  target  tube  {X^X^i.... 

XN)  to  be  reachable  from  the  initial  state  xq,  it  is  sufficient  (but  not  nec- 
— $ 

essary)  that  xqc  Xq.  This  approximation  approach  is  the  basis  for  an  el¬ 
lipsoidal  approximation  algorithm  given  in  the  noct  section,  where  results 
on  the  optimal  control  of  linear  systems  with  quadratic  cost  criteria  are 
used  not  only  to  obtain  ellipsoidal  approximating  sets  but  also  to  derive 
control  laws  which  are  linear. 

4.  An  Ellipsoidal  Approximation  Algorithm 

Consider  the  special  case  of  Problem  3. 1  for  which  the  constraint 
sets  are  the  ellipsoids  described  by: 

Xk  =  {xj^lxj^CJ^CkXk  <  1}  ,  k  =  1,2 . N-l  (3.9) 

XN  =  {xN>xN^xN  -  ^ 


(3.10) 
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Uk  =  {uk|ujtRkuk  <  1}  ,  k  =  0.1 . N-l  (3.11) 

wk  =  {wfc|wjtQkwk  <  O  ,  k=  0,1,...,  N-l  (3.12) 

where  the  matrices^,  R^,  Qk  are  given  positive  definite  symmetric  ma¬ 
trices  for  all  k  =  0,  1,  ....  N-l,  and  the  matrices  Ck  are  given.  We  also 
assume  that  the  matrices  Ak  in  the  system  (3. 1)  are  invertible. 

We  first  internally  approximate  the  set  of  equation  (3.6)  by  an  ellipsoid. 

To  this  end,  we  state  the  following  lemma  the  proof  of  which  can  be  found  in  (Sll 

Lemma  3.1:  Consider  two  ellipsoids  EjtEg  in  Rn  with  support  functions 
flr(x|Ej)  =  (x'Qjx)*^,  ir(x|&2)  =  (x'C^x)*^.  Their  vector  sum  Ej  +  E2 
is  contained  in  the  ellipsoid  E  with  support  function 
<r(x|E)  =  MP^Qj  +<1-P)'1Q2)}1/2 

where  p  is  a  free  scalar  parameter  with  0  <  P  <  1. 

For  ellipsoid' T^  to  be  contained  in  the  set  TN  of  equation  (3.6) 
it  is  sufficient  that  Tj^+G^jWj^jCX^.  The  support  functions  of  the  ellipsoids 

and  Xj^  for  the  case  considered  in  this  section  are,  ir(x|  G^W^j)  = 
(x'Gj^_ jQ^_ jx)^ ^  and  er(x (  Xj^)  =  (x'¥  *x)*^.  By  Lemma  3.  1  if  follows 
that  the  relation  T^  +  Gj^  jW^  j  C  X^  is  satisfied  if  the  support  function  of 
T is  given  by 


Notice  that  if  the  discrete -time  system  (3.1)  results  from  sampling  of  a 
continuous -time  linear  system  the  matrices  will  always  be  invertible. 
However,  it  is  easy  to  see  that  in  what  follows  invertibility  of  the  ma¬ 
trices  A,  is  not  necessary  if  the  matrices  C!  C,  are  positive  definite  for 
all  k.  K  k  k 
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<t(x|Tn)  ■  (x'F^x)1/2 
where  the  matrix  it  given  by 

FN  '  ('-PN»<<'*‘-',NloN.|QN-l0h.|»  lJIJl 

and  is  a  free  parameter  auch  that  #<  <*N  <  '•  If  the  given  conatraint 
aeta  are  auch  that  the  eet  tn  ha  a  a  nonempty  interior,  there  exiata  a 
acalar  PN  with  0  <  oN  <1  auch  that  the  matrix  of  equation  (3. 13)  ia 
poaitive  definite  and  the  ellipaoid 

Tn  =  {x|  x'F N  x  <  1} 


ia  contained  in  the  aet  . 

By  uaing  the  ellipaoid  TN  a  aet  contained  in  the  aet  X*  of  equation 
(3.7)  ia  now  defined  aa  the  aet  of  pointa  xN_j  with  the  property  that  both 


XN-1CN-1CN-1XN -1  -  1 


(3.14) 


and 

*AN-1XN-1  +BN-1uN-1*C  TN  for  Bome  UN-1C  UN-T 

The  aecond  requirement  becomea  in  thia  caae  that 

x'fnx  5.  1  for  aome  uN1  with  j  <  1  (3.15) 


and  with 


x  =  AN-1XN-1  +  BN-1UN-1  (3,16) 

The  aet  of  pointa  j  satisfying  relations  (3.  14),  (3. 15)  and  (3.  16)  clearly 
contains  the  set  of  points  x^  j  with  the  property  that  for  soi  ie  j€  Rm 
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xIm-  1  Ck- 1 CN-  1XN- 1  *  Uk-1RN-1UN-1  *X'FNX  -  1  (J,I7> 

K  *  AN-1XN-1  *  BN-IUN-1  (3.18) 

By  well  known  results  on  the  linear  quadratic  regulator  problem  o (  optimal 
control^^  the  set  of  points  x^_j  satisfying  the  equations  (3.  17)  and  (3.  18) 
for  some  u^jC  Rn  is  given  by 

^N-l  e  txN-l*  x'n-1KN-1xN-1  -  (3.19) 

where  the  positive  definite  matrix  Kn_j  is  given  by  the  discrete  Riccati 
equation 

KN-1  =  AN-1<FN  +  BN-1RN-1BN-1)  1aN-1  +  CN-1CN-1  (3,20) 
Furthermore  a  control  law  which  achieves  reachability  is  given  by 

'‘N-l  s  ►*N-1(xN-1)  =  -<RN-1  +  BN-1FNBN-1)  1 BN - 1 FNAN-1XN-1 

(3.21) 

By  proceeding  with  similar  approximations  we  define  sets 
^N-l'^N-2*  *  ^  *onrl®  ellipsoid  7^  is  empty,  then  the  algorithm 

breaks  down.  This  of  course  does  not  imply  that  the  original  target  tube  is 
not  reachable,  since  approximations  were  involved  in  obtaining  T^.  In  this 
case  if  we  wish  to  proceed  with  the  ellipsoidal  algorithm  we  will  have  to 
start  with  a  "larger"  target  tube  or  "larger"  control  constraint  sets.  We 
summarise  the  algorithm  below: 

An  internally  approximate  modified  target  tube  fx^,  X^,  . . .  ,XN)  and 
effective  target  tube  {Tj,  T^$  ....  T^}  are  given  recursively  by  the  equations 

Xk*  =  {xJx^K^  <  1}  ,  k=l,2,...,N  (3.22) 
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Tk  {x|x'Fkx  <  1)  .  k  *  1,2 . N 


(3.23) 


where 


K 


k-1 


a*  ,(f:1  ♦  bl  .r’^b*  , 1  a.  ,  ♦  c*  ,c.  , 

k-l  k  k-1  k-l  k-1  k-1  k-1  k-1 


kn  t 


(3.24) 

(3.25) 
(3.2b) 


And  the  free  parameter*  Pk>  k  =  1,2,.  .  . ,  N,  are  such  that  0  <  Pk  <  1  and 
the  matrices  F^  are  positive  definite  for  all  k. 

A  sufficient  condition  for  reachability  is  that  the  set 


X* 

o 


(x  |x'K  x  <  1} 
o  o  o  o  — 


(3.27) 


contains  the  initial  state  x  ,  where 

0 


K  =  A'  (F’1  +  B'  R’  1  B  )_1A 
o  o'  1  o  o  o  c 


(3.28) 


Furthermore  a  control  law  that  achieves  reachability  is  given  by  the  equation 

^k(xk)  =  "(Rk  +  BkFk+lBk)  BkFk+lAkxk’  k  =  0,  1,  .  . . ,  N-l  (3.29) 

We  remark  that  another  control  law  that  achieves  reachability  is  the 
control  law  with  a  dead  zone  given  by  equation  (3.29)  if  xjcAJc^k+lAkXk  >  * 
(i.e.  ,  if  A^x^  Tk+j),  and  ^k(xk)  =  0  otherwise.  In  certain  applications  the 
use  of  a  dead  zone  can  be  particularly  beneficial. 

It  should  be  mentioned  that  a  similar  ellipsoidal  algorithm  can  be 
obtained  for  the  case  where  the  given  sets  X^j,  U^,  W^,  k  =  0,  1 ,  .  .  . ,  N-l , 
are  ellipsoids  which  are  centered  at  given  points  other  than  the  origin. 
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The  case  where  the  eyatem  (3.  1)  ia  time-invariant,  the  given 
conatraint  aeta  are  conatant  and  the  final  time  N  approaches  infinity  <  h 
highly  interesting.  The  behaviour  and  the  convergence  properties  of  the 
ellipsoidal  algorithm  under  these  circumstances  will  be  examined  in  the 
next  section. 

5.  Infinite  Time  Behaviour  of  the  Ellipsoidal  Algorithm 

Consider  now  the  case  where  the  system  (3.  1)  is  time  invariant 

xk+i  =  A,k  *  Buk  *  <3-30> 

the  given  constraint  sets  are  constant,  i.e.,  =  R,  =  Q, 

for  all  k,  and  the  final  time  N  approaches  infinity. 

The  ellipsoidal  algorithm  of  equations  (3.  22)  through  (3.26)  under 
these  circumstances,  and  assuming  a  constant  scalar  p  with  0  <  p  <1,  i.e.  , 
Pk  =  P  for  all  k,  is  given  by  the  equations: 

X*  =  {xk|x|Kkxk  <  1}  (3.31) 

Tk  =  {x|x'Fkx  <  1}  (3.32) 

where 

Fk!  =  (1  -  P)(Kk*  -  P"1GQ~1G')  (3.33) 

Kkl  =  A'(Fk!  +  BR'lB')_1A  +¥  (3.34) 

Kn  =  ¥  (3.35) 

under  the  assumption  that  the  matrix  Fk*  is  positive  definite  for  all  k. 

Assume  that  for  some  scalar  (3  with  0  <  p  <  1  the  algorithm  of 
equation  (3.31)  through  (3.35)  possesses  a  positive  definite  steady  state 
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solution  K  given  by 
-oo 

K  A'(F* 1  ♦  BR^B'fU  +V  (3.36) 

-ao-oo 

for  which  the  matrix 

F'*  *  (1  -  P)(K‘‘  -  P^GQ^C)  (3.37) 

•CD  -00 

is  positive  definite.  Then  if  the  initial  state  belongs  to  the  set  X*  = 
(xlx'K^x  <  1}  the  state  of  the  system  (3.  30)  can  be  made  to  stay  indefi- 

_4>  . 

nitely  in  the  tube  (X  ,  X  ,  .  .  .  }  by  application  of  the  linear  time  invariant 
control  law 

p(x)  =  -(R  +  B'F  B)-1BF  Ax  (3.38) 

-00  -oo 

Since  we  will  have  X  C  X  =  {x|x^x  <  l}  infinite  time  reachability  of  the 
given  target  tube  (X,  X,  .  .  .  }  is  achieved. 

It  should  be  noted  that  in  the  actual  operation  of  the  closed-loop 
system  the  initially  given  tube  {X,  X,  ...}  loses  its  significance  since  the 
system  state  will  always  remain  in  the  internal  tube  {X  ,  X  ,  ..  .}  the  sets 
X  of  which  will  differ  significantly  from  the  sets  X  of  the  initial  tube  both 
in  "size"  and  "orientation".  Thus  in  any  infinite  time  design  procedure  the 
given  set  X  and  the  corresponding  matrix  take  the  role  of  a  design  para¬ 
meter  which  can  be  adjusted  to  obtain  different  steady  state  solutions  K_ 
of  the  algorithm. 

A  question  of  importance  is  under  what  circumstances  the  algorithm 
of  equations  (3.33)  through  (3.35)  will  converge  to  a  steady  state  solution 
K  satisfying  the  equations  (3.36)  and  (3.37).  Clearly  given  the  system 
(3. 30)  and  the  matrix  Q  specifying  the  disturbance  constraint  set  W,  the 
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matrices  R  and¥  must  specify  a  sufficiently  "large"  control  constraint 
set  and  a  sufficiently  "large"  target  tube  relative  to  the  site  of  the  dis¬ 
turbance  set  and  the  nature  of  the  matrices  A,  B  and  G  of  the  system.  Thus 
if  the  matrices  R  and  »  specify  relatively  small  constraint  sets  the  algo¬ 
rithm  of  equations  (3.  33)  through  (3.  35)  should  not  be  expected  to  converge 
to  a  steady  state  and  guarantee  reachability  from  some  initial  states.  Now 
in  any  practical  situation  the  designer  is  given  the  system  (3.  30)  and  the 
matrix  Q  specifying  the  set  W  where  the  input  disturbance  belongs,  and 
usually  there  is  a  certain  degree  of  freedom  in  adjusting  the  control  con¬ 
straints,  and  particularly  the  matrix  ¥  specifying  the  target  tube  which 
in  view  of  the  comment  of  the  previous  paragraph  plays  the  role  of  a  de¬ 
sign  parameter.  In  this  sense  a  possible  design  procedure  is  to  initially 
select  the  matrices  R  and  ¥  and  in  case  the  algorithm  does  not  converge 
to  a  steady  state  solution,  to  decrease  these  matrices  by  multiplication  by 
factors  less  than  one  and  repeat  the  procedure  until  convergence  and  satis¬ 
faction  of  the  designer.  It  is  important  however  to  know  under  what  cir¬ 
cumstances  there  exist  matrices  R  and  ¥  such  that  the  algorithm  converges 
to  a  steady  state,  and  furthermore  under  what  conditions  such  matrices  can 
be  obtained  by  repeatedly  multiplying  any  initially  selected  matrices  Rj  and 
^by  factors  of  less  than  one.  This  is  the  object  of  the  next  proposition 
which  states  that  the  design  procedure  outlined  above  is  successful  provided 
the  system  (3.30)  is  stabilizable,  i.e.,  if  there  exists  a  matrix  L  such  that 
the  matrix  (A  -  BL)  is  stable  (has  eigenvalues  within  the  unit  disk  of  the 
complex  plane).  Notice  that  the  system  (3.30)  is  stabilizable  provided  the 
pair  (A,  B)  is  controllable  (but  not  conversely)^0^. 
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Proposition  3.1:  Assume  that  the  system  (3.30)  end  the  positive  definite 
symmetric  matrix  Q  are  given  and  that  the  system  (3.  30)  is  stabilisable. 
Then  given  any  positive  definite  symmetric  matrices  ^  and  Rj  of  appro¬ 
priate  dimension,  there  exists  a  scalar  0  <  p ^  <  1  such  that  for 
every  scalar  p,  0  <  P  <  pj  there  exist  scalars  tj'^l  depending  on  p  such 
that  for  all  matrices^  =  a^,  R  =  bRj  with  0<  a<  Sj,  0<  b  <  b,,  the 
algorithm  of  equations  (3.  33)  through  (3.  35)  converges  to  a  positive  def¬ 
inite  symmetric  steady  state  solution  K  satisfying  equations  (3.36)  and 
(3.37). 

The  proof  of  the  above  proposition  follows  similar,  yet  a  little 
more  complicated,  arguments  with  a  proof  of  convergence  of  usual  Riccati 
equations  to  a  steady  state  solution^0^.  Due  to  its  length  this  proof  will 
be  presented  in  Appendix  11. 

Another  important  question  concerning  the  infinite  time  ellipsoidal 
algorithm  is  whether  the  resulting  linear  time -invariant  control  law  makes 
the  closed-loop  system  asymptotically  stable.  This  question  is  answered 
in  the  affirmative  in  the  following  proposition. 

Proposition  3.2:  Assume  that  the  algorithm  of  equations  (3.33)  through 
(3.35)  converges  to  a  steady  state  solution  K_qo»  where  K  is  a  positive 
definite  symmetric  matrix  for  which  the  matrix  F_qq  of  equation  (3.  37)  is 
also  positive  definite.  Then  the  closed-loop  system  resulting  from  appli¬ 
cation  of  the  linear  time-invariant  control  law  of  equation  (3.38)  is  asymp¬ 
totically  stable. 


The  proof  of  the  above  proposition  will  also  be  given  in  Appendix  II. 
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An  immediate  consequence  of  the  above  proposition  is  that  tran¬ 
sients  due  to  initial  states  will  vanish  eventually  during  the  operation  of 
the  closed-loop  system.  More  accurately  for  any  e  >  0  it  can  be  guar¬ 
anteed  that  after  a  sufficient  number  of  steps  the  state  will  be  confined 
in  the  set  +  <B,  where  B  is  the  unit  ball  in  Rn,  and  this  will  occur 
for  any  initial  state  xq  in  Rn. 

6.  Discussion  and  Sources 

The  problem  of  the  reachability  of  a  target  tube  was  examined  in 
this  chapter  with  emphasis  in  the  development  of  an  ellipsoidal  approxima¬ 
tion  algorithm  that  appears  to  have  potential  for  practical  applications. 

The  attractive  feature  of  the  ellipsoidal  algorithm  is  that  it  pro¬ 
vides  a  linear  control  law  which  in  the  infinite  time  case  makes  the  closed- 
loop  system  asymptotically  stable.  Furthermore  for  the  infinite -time 
case  the  existence  result  of  Proposition  3. 1  guarantees  that  the  algorithm 
is  applicable  to  every  linear  time -invariant  system  which  is  stabilizable. 
Thus  the  ellipsoidal  algorithm  appears  to  offer  practical  advantages  as  a 
design  method  for  many  regulation  and  tracking  problems  which  involve  a 
linear  system,  and  for  which  the  statistics  of  the  uncertain  quantities  are 
unknown  and  difficult  to  measure,  or  for  which  specified  tolerances  must 
be  met  with  certainty. 

A  number  of  questions  concerning  the  performance  of  the  algorithm 
remain  as  yet  unresolved.  One  such  question  concerns  the  quality  of  the 
approximation  involved  in  the  algorithm*  If  appears  to  be  very  difficult 
to  obtain  precise  estimates  of  the  approximation  which  are  applicable  to 
large  classes  of  systems.  Thus  some  further  research  and  simulations 


are  required  in  thia  area.  Another  question,  and  in  the  author1*  opinion 
the  most  important,  touches  upon  the  merits  of  the  whole  minimax  design 
philosophy.  Minimax  designs  are  in  general  concervative,  optimal  against 
the  worst  case.  In  the  particular  case  of  the  ellipsoidal  algorithm  the 
result  is  that  the  feedback  gains  of  the  controller  tend  to  be  large  in  mag¬ 
nitude,  a  feature  which  in  some  cases  may  be  undesirable.  Furthermore 
this  situation  is  adversely  effected  by  the  approximations  involved.  Only 
simulations  and  practical  experience  can  give  some  answers  to  this  ques¬ 
tion. 

Many  of  the  results  of  this  chapter  have  been  reported  in  (Bl).  The 
approuch  used  in  this  reference  is  purely  geometrical  and  is  applicable  to 
a  large  class  of  problems.  It  not  required  that  the  system  is  linear  and 
that  the  given  sets  are  closed,  convex  or  compact.  In  fact  not  even  the 
linear  vector  space  structure  of  the  space  of  definition  of  the  system  is 
necessary.  However  the  ellipsoidal  algorithm  is  applicable  only  to  the 
class  of  problems  considered  in  this  chapter.  The  Propositions  3.  1  and 
3. 2  have  not  appeared  earlier.  It  is  interesting  to  note  that  the  equations 
of  the  ellipsoidal  algorithm  are  very  similar  to  Riccati  equations  related 
to  linear  multistage  games  with  quadratic  cost  functional^^*).  In  fact 
the  proofs  of  Propositions  3. 1  and  3.2  were  to  a  large  extent  motivated 
by  this  similarity. 

The  problem  of  the  reachability  of  a  target  set  is  the  special  case 
of  the  Problem  3. 1  where  the  sets  for  all  k  except  k  =  N  are  equal  to 
the  whole  space  Rn.  This  problem  for  a  linear  discrete-time  system, 
closed-loop  control  and  perfect  state  information  was  first  considered 
by  Witsenhausen^^'^^  in  the  framework  of  a  more  general  minimax 
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control  problem.  The  lime  problem  for  a  continuous -time  system  but 
an  open-loop  controller  was  also  considered  by  Delfour  and  Mitter  in  (Del). 

Problems  related  to  those  of  this  chapter  that  require  attention 
are  the  case  of  a  nonlinear  system  and  the  case  of  a  continuous,  linear 
or  nonlinear,  system.  The  results  in(Bl)  ure  applicable  to  nonlinear 
discrete -time  systems  however  no  practical  algorithms  applicable  to 
nontrivial  systems  are  available  at  this  moment.  The  case  of  a  continuous¬ 
time  linear  system  is  considerably  more  difficult  to  handle  than  the  case 
of  a  discrete-time  system.  Some  results  obtained  by  the  author  in  this 
area  are  not  as  yet  conclusive. 

Finally  we  note  that  the  problem  of  the  reachability  of  a  target  tube 
with  imperfect  state  information,  including  the  case  where  instead  of  the 
entire  state  only  a  linear  output  of  the  system  is  measured  exactly,  will  be 
considered  in  Chapter  6. 


CHAPTER  4 


STATE  ESTIMATION  PROBLEMS  FOR  A  SET 
DESCRIPTION  OF  THE  UNCERTAINTY 

1 .  General  Remarks 

In  this  chapter  we  digress  from  minimax  control  problems  in  order 
to  consider  some  state  estimation  problems  which  involve  a  set-membership 
description  of  the  uncertainty.  Such  problems,  though  important  in  their 
own  right,  are  essential  for  the  solution  of  minimax  control  problems  with 
imperfect  state  information.  Although  the  concepts  to  be  presented  are 
applicable  to  much  more  general  situations  we  will  be  concerned  exclusively 
with  the  case  of  a  linear  discrete-time  dynamic  system 

xk+l  =  Akxk  +  Bkwk  »  k  =  0,  1,  ... ,  N-l  (4..1), 

to  which  there  are  available  noise-corrupted  measurements 

zk  =  ^"kXk  *  vk  (4*2) 

n  r 

where  x^e  R  is  the  system  state,  w^€  R  is  an  input  disturbance  vector  and 

v,  e  rP  is  the  measurement  noise  vector.  We  assume  that  there  is  no  control 
k 

input  to  the  system.  The  algorithms  that  we  derive  however  can  be  trivially 
modified  to  take  into  account  the  effect  of  any  known  deterministic  input  by 
virtue  of  the  linearity  of  the  system  and  measurement  equations. 

In  a  stochastic  estimation  problem  involving  the  system  (4.1)  with 
the  measurements  (4.2)  the  uncertain  quantities,  i.  <■  ,  the  initial  state  and 
the  input  and  n  .  >urement  noise  vectors,  are  modelled  as  mutually  inde¬ 
pendent  randorr  vectors  with  known  probability  density  unctions.  In  this 
case  all  information  about  the  system  state  at  any  time  that  is  provided  by 
the  measurements  is  contained  in  the  probability  density  function  of  the  state 
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conditioned  on  these  measurements.  This  conditional  probability  density 
function  is  then  used,  explicitly  or  implicitly,  to  determine  an  estimate 
of  the  system  state  which  is  best  in  some  prescribed  sence. 

In  the  case  considered  here,  the  uncertain  quantities  are  not  mod¬ 
elled  as  random  vectors,  but  are  considered  instead  to  be  unknown  except 
that  they  belong  to  given  subsets  of  appropriate  vector  spaces.  Under 
these  circumstances,  all  information  about  the  system  state  at  any  time 
that  is  provided  by  the  measurements  may  be  summarized  in  t.\e  set  of  all 
states  consistent  with  both  the  measurements  received  and  the  constraints 
on  the  uncertain  quantities.  Once  this  set  of  possible  states  is  characterized 
a  point  estimate  can  be  selected  using  some  criterion  such  as  the  minimax 
error  criterion  for  example.  In  what  follows  however  we  will  be  concerned 
exclusively  with  the  characterization  of  the  set  of  possible  states  or  some 
approximation  thereof.  Since  for  the  special  cases  that  we  will  consider  this 
set  will  be  an  ellipsoid,  if  a  point  estimate  is  desired  the  center  of  the  ellip¬ 
soid  is  the  natural  candidate. 

Two  distinct  types  of  constraints  on  the  uncertain  quantities  will  be 
considered.  The  first  is  the  energy-type  constraint 
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where^,  Q^,  are  given  positive  definite  symmetric  matrices  for  all  k. 
For  this  constraint  we  show  that  the  set  of  possible  states  at  any  time  con¬ 
sistent  with  the  output  measurements  is  an  ellipsoid  whose  center  and 
weighting  matrix  are  generated  by  equations  identical  to  those  associated 
with  the  best  linear  estimator  (Kalman  filter)  fora  certain  stochastic  esti- 


i 


-68- 


mation  problem.  We  shall  demonstrate  a  one-one  correspondence  be¬ 
tween  estimation  problems  where  the  uncertain  quantities  satisfy  an  energy 
constraint  and  linear  minimum  variance  stochastic  estimation  problems. 

Once  this  correspondence  is  established  we  will  be  able  to  use  available 
results  in  stochastic  estimation  theory  to  derive  estimators  for  the  energy 
constraint  case  for  a  number  of  problems  of  interest  including  the  filtering, 
prediction  and  smoothing  problems. 

The  second  type  of  constraint  that  we  consider  is  the  more  practically 

important  case  where  the  uncertain  quantities  are  constrained  at  each  instant 

of  time  to  lie  in  ellipsoids,  i.e. , 

-1  , 

x*  ¥x  <  1 ,  w!  ,Q,  .  w,  .  <  1,  v!  R.  v.  <  1,  k  =  1 , 2,  .  . ,  N 
o  o  —  k-1  k-1  k- 1  —  k  k  k  —  *  ’ 

In  this  case  the  set  of  states  consistent  with  the  measurements  is  not  an 
ellipsoid  and  it  is  not,  in  general,  characterized  by  a  finite  set  of  numbers. 
However,  an  ellipsoid  bound  to  it  can  be  detexmined  by  bounding  the  instant¬ 
aneous  constraints  by  an  energy  constraint  and  using  the  results  derived  for 
that  constraint.  The  resulting  estimator  for  the  case  of  the  filtering  and  the 
prediction  problem  is  similar  to  that  proposed  by  Schweppe'  7  but  it  has 
two  important  advantages:  first  the  gain  matrices  do  not  depend  on  the 
particular  measurements  received  and  are  therefore  precomputable  and, 
second,  it  reduces  to  a  constant  system  as  the  final  time  beccmes  infinite. 

In  all  other  respects  it  is  comparable  to  that  proposed  by  Schweppe.  Further¬ 
more  our  approach  permits  the  derivation  of  an  estimator  for  the  smoothing 
problem  which  has  not  been  previously  considered  in  the  literature. 
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2.  Formulation  of  the  Problem  with  an  Energy  Constraint 

In  this  section  we  formulate  a  general  estimation  problem  involving 
a  linear  discrete-time  dynamic  system  and  a  combined  energy  constraint 
on  the  uncertain  quantities.  This  problem  includes  as  special  cases  the 
filtering,  prediction,  and  smoothing  problems. 

Problem  4.1:  Consider  the  linear  discrete-time  dynamic  system 
xk+l  =  AkXk  +  BkWk  *  k  =  0,  1, .  . . ,  N- 1 

to  which  there  are  available  noise -corrupted  measurements 

zk  =  ckxk +  vk 

where  Rn  is  the  system  state,  w^e  Rr  is  the  input  disturbance  vector, 
VrP  is  the  measurement  noise  vector,  and  the  matrices  A^,  B^,  have 
the  appropriate  dimensions.  The  initial  state  xq  and  the  disturbances  w^, 
are  assumed  unknown  except  that  they  satisfy  the  energy  constraint 

-IN.  . 

v*o  +  kz1(wk-iQk-iwk-i  +vkRk -  1  (4,3) 

where  j.R^,  k  =  1,  2,  .  . ,  N,  are  given  positive  definite  symmetric 

matrices.  Let  i,  k  be  arbitrary  integers,  0<  i  <  N,  0  <  k  <  N.  It  is  re¬ 
quired  to  find  the  set  of  possible  system  states  x.  at  time  i  which  are 
consistent  with  the  constraint  (4. 3)  and  the  measurements  z  j ,  z^,  . .  . ,  z^ 
up  to  time  k. 

We  remark  that  if  i  =  k  this  problem  is  usually  called  the  filtering 
problem,  if  i  >  k  it  is  called  the  prediction  problem,  and  if  i  <  k  it  is  called 
the  smoothing  problem. 
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In  the  next  section  we  will  obtain  a  general  solution  to  the  above 
problem  by  associating  it  with  a  stochastic  estimation  problem,  the  solution 
of  which  is  wexl  known.  We  will  then  use  this  general  solution  to  obtain 
estimators  for  the  special  cases  of  filtering,  prediction,  and  smoothing. 

3.  A  General  Solution  to  the  Problem  with  an  Energy  Constraint 

Given  any  estimation  problem  where  the  uncertain  quantities  are 
unknown  except  that  they  lie  in  some  given  set  it  is  possible  to  give  a  precise 
characterization  of  the  set  X.^  of  possible  states  x^  at  time  i  consistent  with 

the  measurements  Zj, :  . . z^  in  terms  of  the  given  constraint  set  and 

the  system  and  measurement  equations.  This  characterization  is  usually 
quite  elaborate  but  for  the  Problem  4. 1  it  takes  a  particularly  useful  form. 

A  great  deal  of  insight  can  be  obtained  through  it,  and  most  importantly  it 
leads  to  a  direct  correspondence  with  linear  minimum  variance  stochastic 
estimation  problems.  We  will  first  introduce  some  notation. 

Let  uc  pn+N(r+p)  tjje  vector  consisting  of  all  the  uncertain  quan¬ 
tities  according  to  the  relation 

u  =  (x^,  w^,  w'j, . . . ,  wJ^  j,  .  . . ,  vjj)'  (4.4) 

Let  us  also  combine  all  measurements  received  up  to  time  k  into 
one  vector 

^k  =  **1’  z2*  ’  *  ’  *  *k^'  (4-  5) 

Both  the  state  of  the  system  x^  at  any  time  i,  and  the  vector  can 
be  obtained  from  the  vector  u  of  equation  (4.4)  by  a  linear  transformation 

x.  =  L.u  (4.6) 

ii 
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D.  u 
k 


(4.7) 


where  the  n  x  (n  +N(r+p)]  matrix  L.  and  the  kp  x  (  n  +  N(r+p))  matrix 
are  given  for  all  i  and  k  by 


Lj  =  [4>(i,0),$(i,  1)Bq . ♦(i,i-l)H_2,B._1,0 . 0]  (4.8) 


Dk  = 


Cj^l.O),  C1B0.  0,0,...,  0  0,  .  . ,  0  I,  0,  .  .,0  0,  .  . ,  0 

C2<®(2,  0),  C2$(2,  1)Bo,  C^Bj,  0,  ....  0  0,  .  . ,  0  0,1,  0,..,0  0,  .  . ,  0 


Ck$(k,0)  Ck®(k,  l)Bo,  Ck<X\k,2)B1,..,CkB^_1,  0,  ...0  0,  ...0,1  0,  ...0 

(4.9) 


where  the  transition  matrix  $(i,j)  is  given  by 


♦  (i.j)  =  Ai-1Ai-2‘  •  *Aj  for  j  <  i 

•(U)  =  I 


and  where  the  dimensions  of  the  aero  and  identity  matrices  in  the  above 
equations  are  consistent  with  the  multiplications  indicated  in  equations  (4.6) 
and  (4.7). 

The  energy  constraint  (4.3)  implies  that  the  vector  u  of  equation  (4.4) 
belongs  to  the  ellipsoid 

U  =  {u|u'M’1u<  1}  (4.10) 


where  the  positive  definite  matrix  M  is  defined  as 


M  = 


(4.11) 


0 
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Now,  for  fixed  measurements  G^.  the  set  X.  ^  which  solves  the 


Problem  4.  1  can  be  conveniently  characterized  as 
Xi|k  =  *xlx=  Liu*  =  Dku*  ucU^ 


(4.12) 


By  defining  the  set  of  all  possible  vectors  u  consistent  with  the  mea¬ 
surement  vector  G, 


u.  =  {u  I  G.  =  D.u,  ue  u} 


(4.13) 


we  have  from  equation  (4.12)  that  the  set  X^|  ^  is  given  by  the  equation 


X...  =  L.U. 
i  k  i  k 


(4.14) 


Thus  the  set  X.  ^  can  be  obtained  by  a  linear  transformation  on  the 
A 

set  which  is  the  set  intersection  of  the  set  U  with  the  linear  variety 
(manifold)  (u|  G^  -  D^u}  defined  by  the  measurements.  Since  the  set  U  is 
an  ellipsoid  in  the  space  pn+N(r+P)  the  set  intersection  is  also  an  ellip¬ 
soid  and  the  set  X.  is  also  an  ellipsoid  since  by  equation  (4.  14)  it  is 
obtained  through  a  linear  transformation  on  an  ellipsoid.  We  proceed  to 
characterize  the  center  and  weighting  matrix  of  the  ellipsoid  X^  in  the 


following  proposition. 


Proposition  4. 1 :  Thi  ellipsoid  X.  which  selves  Problem  4. 1  is  given  for 
all  i,  k,  0  <  i  <  N,  0  <  k  <  N  by 


xUk  1  tx|  <x-i,|k>'2lik(x-Ji|k)  i  1  -i2<k» 


(4.15) 


where  the  matrix  is  given  by 

r.|k  -  L,[M-  MD'ID^f'D^L; 

the  n-vector  x.  is  given  by 


(4.16) 


w 
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A 


Ei|k  '  ~i - k'~k - k'  *k 

and  the  nonnegative  scalar  6^(k)  is  given  by 

62(k)  =  S^MD^)-1  Ck 


(4.17) 


(4.18) 


In  the  case  where  the  matrix  ZL|k  is  only  positive  semidefinite  but  not  positive 
definite  the  ellipsoid  X.|k  is  characterized  by  its  support  function 

<r(x*|X.|k)  =  <x*,^i|k>  +  (l-62(k)]1/2(x*!L.|kx*)1/2  (4.19) 

It  should  be  mentioned  that,  as  will  be  shown  later,  the  matrix  E. is  invert- 

i|k 

ible  provided  the  matrices  Ak  in  the  system  equation  (4. 1)  are  invertible. 
Proof:  Since  the  equation  (4, 19)  implies  the  equation  (4. 15)  whenever  the 
matrix  E^  invertible,  it  will  be  sufficient  to  prove 

a(x*  |Xi|k)  =  sup  <x*,x>  =  <x*4|k>+  [  1  -  62(k)]  1/2(x*'E.jkx*)1/2 

A 

We  will  first  characterize  the  support  function  of  the  ellipsoid  Uk  of  equation 
(4.14) 

Uk  =  {u|CR  =  Dku,  u'M_1u<  1}  (4.14) 

Consider  the  space  pn+N(r+p)  norm 

||u||  =  (u'M^u)1^2 

With  this  norm  the  set  U  =  {u|u'M"*u  <  1}  becomes  the  unit  ball  in  pn+N(r+p) 

A 

and  the  set  Uk  of  equation  (4. 14)  is  the  intersection  of  the  unit  ball  with  the 
linear  variety  (u|  =  Dku}.  Let  be  the  (unique)  vector  of  minimum 

norm  on  this  linear  variety  given  by  the  projection  theorem^1**) 

K  .  MD-(DkMD')-‘  {k 


(4.20) 
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It  can  be  seen  from  equation  (4.  9)  that  the  matrix  has  full  rank  and 
therefore  the  matrix  (D^MDJ^)  is  invertible  thus  justifying  the  notation 
used  above. 

A 

The  set  Uk  is  now  given  by: 

fjR  =  {u|  ||u-uk||2  <  l-||uk||2,  tk  =  Dku} 


and  can  be  also  characterized  as 

Uk  =  uk  +  (l-||fik||2)1/2,Sk 


where  Uk  is  the  intersection  of  the  unit  ball  with  the  nullspace  N(Dk)  of  the 
matrix  Dk>  From  the  above  equation  we  have  for  the  support  function  of  (jk 


er(u*|U.)  =  <u*.  u.>  +  (1  -  ll&Jl2)172  sup  <u*,u>  (4.21) 

k  k  k  ||u||2<l 

ucNfDk) 


By  using  Theorem  5.&.1  in  (Lul)  we  have 


sup  <u*,  u>  =  ||u* 

lluil2<  i 

ueN(5k 


A*  ♦ 

where  u  is  the  projection  of  the  vector  u  on  the  subspace  N(Dk).  Using 
again  the  projection  theorem  we  obtain 


110*11  =  {u^M-MD^MD^f^Mlu*}172 


Using  this  relation  in  (4.21)  we  have 

<r(u*|0k)  =  <u*,  uk>  +  (1  -  ||uk||2)1/2{u*,(M-MDjc(DkMDjc)"1DkM]u*}1/2 

A 

Using  the  fact  that  X.|k  =  in  the  above  equation,  we  have 


1 

I 
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o-(x*|x.jk)  =  <r(L!.x*|Uk) 

=  <x*,L.(ik>  +  (1  -  |j^||2)1/2{?c*,Li[M-MDjc(DkMDjc)"1DkM]Llx*}1/2 

Now  from  equations  (4. 16),  (4. 17),  (4  .1 8)  and  (4.  20)  we  obtain 
<T(x*jX.|k)  =  <x*,^|k>  +  [l-62(k)]1/2(x*'Zi(kx*)1/2 

which  was  to  be  proved.  Q.E.D. 


The  equations  (4.17)  and  (4. 16)  for  the  center  x.  |k  and  the  weighting 
matrix  £.|k  of  the  ellipsoid  X.  ^  appear  to  quite  formidable  in  view  of  the 


ilk 


complicated  expressions  (4.  8)  and  (4.9)  for  the  matrices  and  Dk>  Y  *  we 
will  be  able  to  obtain  efficient  recursive  algorithms  for  the  computation  of 
$.|k  and  Z.|k  by  associating  the  Problem  4.  1  with  the  following  linear  mini¬ 
mum  variance  estimation  problem. 


Problem  4 . 1 1 :  Consider  the  Problem  4.  1  where  the  vectors  xq,  wq,  Wj,  .  . ,  wN  j, 
v2*  '  *  ’  VN  inste*d  of  satisfying  the  energy  constraint  (4.  3),  are  independent 
random  vectors  with  zero  mean  and  covariances 


E{j 


i> 


E{ 


E{v.vl}  = 


i  =  1.2, 


N 


Find  the  linear  minimum  variance  estimate  x^|k  of  the  system  state  x.  at  time 
i  given  the  measurements  z^,  z^,  . . . ,  zk  and  also  find  the  covariance  of  the 
estimation  error 

ri|k  *  E{<vJi|k)<ItrSi|k,'} 


By  using  the  relations  (4.  6)  and  (4.7)  it  can  be  seen  that  the  above 
problem  is  equivalent  to  finding  the  linear  minimum  variance  estimate 
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A 

x. of  the  vector  x. 
i|k  1 


x.  =  L.u 
1  l 


(4.6) 


given  the  measurement 

K  =  Dku 


(4.7) 


where  u  is  a  zero  mean  random  vector  with  covariance  M  given  by  equation 
(4.11).  The  solution  of  this  problem  is  well  known  and  given  in  many  sources 
(Lul),  (Brl)'  estimate  £  is  given  by  equation  (4.17),  i.e.,  by  the 

same  expression  as  the  center  of  the  ellipsoid  X.^  in  Proposition  4.  1.  The 
covariance  matrix  21 is  given  by  equation  (4.  16),  the  same  expression  which 
gives  the  weighting  matrix  of  the  ellipsoid  X^  in  Proposition  4.  1.  Thus 
there  is  a  one-one  correspondence  between  Problem  4.  1  and  the  stochastic 
estimation  Problem  4.  1*  which  is  reflected  in  identical  expressions  for  the 
center  x.^  and  weighting  matrix  £  of  the  ellipsoid  X.  )u  on  one  hand,  and 


1|k 


i|k 


the  linear  minimum  variance  estimate  x.i.  and  error  covariance  £„  in 

i|k  i|k 

Problem  4.1*  on  the  other.  Now  from  the  well  known  results  in  stochastic 

estimation  theory  the  estimate  xl  and  error  covariance  are  computed 

by  efficient  recursive  algorithms  (Kalman  estimators)  which  do  not  require 

storage  of  the  measurements.  The  same  algorithms  are  applicable  and  can 

be  used  for  obtaining  the  center  x^  and  weighting  matrix  of  the  ellipsoid 

X.i,  -solution  of  Problem  4.1. 
i|k 

2 

Concerning  the  scalar  6  (k)  of  equation  (4. 18)  the  following  recursive 
relation  can  be  proved  for  all  k 

6  (k)  =  6  (k-1)  +  (zk-CkAk  lxk  l  jk_  j  )'(Ckskjk_1C^+Rk) 


(4.22) 


r 


This  relation  can  be  used  to  calculate  the  scalar  6  (k)  recursively  without 
requiring  storage  of  all  the  measurements  up  to  time  k.  The  equation  (4.22), 
the  continuous  counterpart  of  which  has  been  proved  in  (B2),  can  be  proved 
in  a  number  of  ways.  One  possible  method  is  by  direct  manipulation  from 
the  equation  (4.  18)  using  the  equations  (4.  8),  (4.  9)  and  (4.  11).  This  method 
is  straightforward  but  too  lengthy  and  tedious  to  be  profitably  displayed 
here.  Another  method  to  prove  the  equation  (4.22)  is  by  considering  the 

filtering  case  of  Problem  4. 1  and  by  casting  it  as  an  optimal  tracking  prob- 

•  •  ^ 

lem  as  was  done  in(B2).  The  equation  (4.22)  follows  directly  from  the  solu¬ 
tion  of  this  tracking  problem. 

The  preceding  discussions  have  demonstrated  that  the  ellipsoid  dis¬ 
solution  of  Problem  4.  1  can  be  characterized  from  Proposition  4.  1  by  using 
results  of  stochastic  estimation  theory  (Kalman  estimators)  for  the  recursive 

computation  of  the  center  x^  and  the  weighting  matrix  E.  ^  and  by  using 

2 

equation  (4.22)  for  the  computation  of  the  scalar  6  (k).  In  the  next  section 
we  shall  explicitly  characterize  the  solution  of  the  Problem  4. 1  for  the  case 
of  the  filtering  problem. 

We  finally  note  that  the  correspondence  between  the  Problems  4. 1 
and  4. 1'  can  be  extended  to  some  related  problems  not  explicitly  considered 
here.  Such  is  the  problem  where  there  is  no  error  in  the  measurement 
equation  (4.2),  i.e.,  z^  =  In  this  case  the  energy  constraint  (4.3) 


becomes 


l  n  i 

x'^  x  +  S  w!  ,Q.  ,w.  .  <  1 

O  O  1  1-1  1-1  1-1  — 


The  Proposition  4. 1  can  be  easily  shown  to  hold  with  the  matrices  L.,  D,  ,  M 

X  K 
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appropriately  modified.  Under  these  circumstances  however  the  matrix 

need  not  have  full  rank  and  consequently  the  matrix  (D^MDp  may  not 

be  invertible.  In  this  case  it  can  be  proved  similarly  as  in  Proposition  4. 1 

that  the  center  x.|^  of  the  ellipsoid  X^.  the  weighting  matrix  and  the 
2 

scalar  6  (k)  in  equation  (4.19)  are  given  by 

zih-  Li<M  -  skDkM>L! 

where  the  matrix  is  any  solution  of  the  equation 
skDkMDk  ■  MDk 

and 

‘ilk  ■  LiMDk*k 
62(k)  = 

where  the  vector  y^  is  any  solution  of  the  equation 

Wk  *  £k 

The  correspondence  with  a  stochastic  estimation  problem  similar  to  Problem 

4.  1'  which  involves  no  measurement  noise  can  still  be  established  and  the 

(Tl)  (,T2) 

results  for  this  problem''  J  can  be  used  for  the  solution  of  the  esti¬ 

mation  problem  with  an  energy  constraint  but  no  measurement  noise. 

4.  Filtering  for  the  Case  of  Energy  Constraints 

In  this  section  we  will  utilize  the  general  solution  of  Problem  4. 1 
as  given  by  Proposition  4. 1  and  the  one-one  correspondence  with  the  sto¬ 
chastic  estimation  Problem  4. 11  that  was  demonstrated  in  the  previous 
section  to  write  down  explicitly  and  in  recursive  form  the  solution  for  the 
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filtering  cate.  Entirely  similar  equations  can  be  written  for  the  prediction 
and  smoothing  cases(®2^’  (^a*)  Although  it  is  possible  to  write  the 

solution  for  the  general  case'  '  for  simplicity  we  will  assume  in  this  and 

subsequent  sections  that  the  matrices  in  the  system  (4. 1)  are  invertible 
for  all  k.  This  assumption  will  guarantee  the  existence  of  all  the  inverses 
that  will  appear  in  the  expressions  that  follow. 


Proposition  4.2:  The  solution  of  Problem  4. 1  in  the  filtering  case  is  the 
ellipsoid  Xji  ^  given  for  all  k,  0  <  k  <  N,  by  the  equation 


xk|k  ■  {xl<x-4k,'2k|k(jt-ik)  ± 


(4.23) 


where  the  positive  definite  symmetric  matrix  E^|  ^  is  given  recursively  by 
the  Riccati  equation 


siU  ■  v'ih-i +  ciR.:'ci''1 


(4.24) 


E. 


i-1 


=  A.  | .  .A!  ,  +  B.  .Q.  .B!  . 

l-l  l-l  I  l-l  l-l  l-l  l-l  l-l 


(4.25) 


.|o  *♦ 


(4.26) 


the  vector  x^  is  the  solution  of  the  equation 

x.,,  =  A.x.  +  E.  .  i .  .  C! ,  JR!-?,  (z. , .  -  C. ,  .A.x.) 
1+1  1  1  1+1  1+1  1+1  1+1'  1+1  1+1  1  v 


(4.27) 


A 

X 


(4.28) 


and  the  nonnegative  scalar  6  (k)  is  given  by  the  equation 


«2(k)  =  E  (z.-C.A.  .x.  .  )'(C.E.  | .  .C! +R.)’1(z.-C.A.  ,x.  .) 
'  '  ._j  i  i  i-l.a-1'  '  l  i  1 1—  1  l  l  '  i  i  i-1  i-l7 


(4.29) 
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Proof;  The  proof  follows  directly  from  Proposition  4. 1  for  i  =  k,  by  uti¬ 
lizing  the  correspondence  with  the  stochastic  estimation  Problem  4. 11 
demonstrated  in  the  previous  section,  and  by  using  also  equation  (4.22). 

5.  Formulation  of  the  Problem  with  Instantaneous  Constraints 

While  the  preceding  sections  show  it  to  be  of  theoretical  interest, 
the  model  for  the  uncertainty  described  by  the  energy  constraint  (4.  3)  is 
of  limited  use  as  far  as  practical  applications  are  concerned.  A  situation 
which  appears  more  often  in  practice  is  that  in  which  the  uncertain  quan¬ 
tities  are  individually  constrained  at  each  point  in  time.  In  this  section 
we  formulate  such  a  problem  which  is  then  solved  in  Sections  6  and  7  using 
the  results  of  the  preceding  sections.  In  particular,  we  bound  the  instant¬ 
aneous  constraints  by  a  single  combined  energy  constraint  and  apply  the 
results  of  Section  4.  We  concentrate  our  attention  to  the  filtering  case. 
Similar  estimators  can  be  derived  for  the  prediction  and  smoothing  prob¬ 
lems  by  using  the  same  approach.  The  resulting  estimator  is  shown  to  be 
simpler  but  otherwise  comparable  to  the  one  proposed  by  Schweppe'  ' 
with  the  additional  advantage  that  it  possesses  a  steady-state  structure. 

Problem  4.2:  Consider  Problem  4.  1  in  which  the  single  energy  constraint 
(4.3)  on  the  uncertain  quantities  is  replaced  by  the  three  individual  instant¬ 
aneous  constraints 


x'^x  < 
o  o  — 

1 

(4. 30a) 

wkQk‘wk  i. 

1. 

k  =  0,  1,  ...N-l 

(4. 30b) 

vkRklyk  - 

1. 

k  =  1,2,  ..,N 

(4.30c) 
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where¥,  Q^,  R^  are  positive  definite  symmetric  matrices.  As  in  Prob¬ 
lem  4. 1,  find  the  set  of  system  states  at  time  i  that  are  consistent 
with  both  the  measurements  z^,  z^,  . .  . ,  z^  up  to  time  k  and  the  constraints 
(4.30). 


6.  The  Filtering  Problem  with  Instantaneous  Constraints 

Contrary  to  the  case  of  energy  constraints,  it  is  very  difficult  io 
obtain  the  exact  solution  of  Problem  4.2.  As  mentioned  earlier  the  energy 
constraint  (4.  3)  defines  an  ellipsoid  in  the  space  pn+N(r+P)  Since  the 
measurements  z  j,  z^, . . ,  z.  define  a  linear  variety  in  this  space  and  since 
the  intersection  of  an  ellipsoid  with  a  linear  variety  is  also  an  ellipsoid,  the 
set  of  possible  system  states  X.  j^,  obtained  by  a  linear  transformation  on 
this  ellipsoid  intersection,  is  also  an  ellipsoid,  as  found  in  Sections  3  and  4. 
The  individual  instantaneous  constraints  (4.30)  do  not,  on  the  other  hand, 
define  an  ellipsoid,  and  thus  the  intersection  of  the  linear  variety  defined 
by  the  observed  measurements  with  the  subset  of  pn+N(*+p)  satisfying  (4.  30) 
is  not  in  general  an  ellipsoid.  Consequently,  the  set  of  system  states  at 
time  i  consistent  with  the  measurements  Zj,  z^,  •  . ,  is  not  in  general  an 
ellipsoid:  it  is  a  convex  set  that,  in  contrast  to  the  ellipsoidal  case,  cannot 
in  general  be  characterized  by  a  finite  set  of  numbers. 

Thus  one  is  forced  to  seek  approximate  solutions  to  Problem  4.2. 

(SI ) 

The  approach  taken  by  Schweppe'  '  is  to  compute  a  bounding  ellipsoid  to 
the  set  X^|k«  Since  an  ellipsoid  in  Rn  is  completely  characterized  by  an 
n- vector  (its  center),  and  an  nxn  weighting  matrix,  the  storage  problem  is 
reduced  to  more  manageable  proportions.  Schweppe  considered  the  filtering 
and  prediction  problems  for  a  discrete -time  system  in  (SI),  and  gave  a 
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recursive  algorithm  for  the  center  and  weighting  matrix  of  a  bounding 

ellipsoid  to  the  set  of  possible  states.  The  approach  used  was  to  bound 

recursively  the  set  of  possible  states  at  each  time  instant  by  an  ellipsoid. 

This  algorithm  was  later  extended  to  a  continuous -time  system  using  a 

IS2) 

discrete-to-continuous  limiting  argument.  '  The  following  lemma  gives 
the  filtering  algorithm  that  is  presented  by  Schweppe  in  (SI). 


Lemma  4.  1 :  A  bounding  ellipsoid  to  the  set  of  system  states  of  Prob¬ 

lem  4.2,  is  given  for  all  k,  0  <  k<  N,  by: 

X*|k  =  {x|(x-Jk)'^fk(x-Jk)  <  1} 

where  the  positive  definite  matrix  i®  given  recursively  by  the  equations 


si!i  ■  ♦«ici*;1clrI 


(4.32) 


Vi  •  |i-iAi-i  +pi-iBi-iQi-iB!-i  <4-33> 


£  ,  =¥ 
o  o 


(4.34) 


the  vector  x^  is  the  solution  of  the  equation 

xi+l  =  Aixi  +  pi+l(1  '  6i+l*  Zi+l|i+lC!+lRi+l(*i+rCi+lAixi^ 


(4.35) 


with  the  initial  condition 


A  A 

x  =  0 

o 


(4.36) 


t2  , 


and  the  nonegative  sc&lar  6.  is  given  for  all  i  by 

62  =  (r.-C.A.  J'fd-p.r^.Su  ,C!  +  P T 1 R .J  _1(z.-C.A.  .) 

1  'll  i-l  i-l'  *'  r  1  i[i-I  1  K1  lJ  'll  1-1  1-1' 


(4.37) 
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where  the  scalars  j,  p^,  are  free  parameters  with  0  <  (3^  j  <  1, 
0  <  p.  <  1,  i  =  1, 2,  .  . .  ,N. 


The  estimator  of  the  above  lemma  has  the  same  basic  structure  as 
the  stochastic  Kalman  filter.  It  should  be  noted,  however,  that  the  gain 
matrix  Pitld  -  *2 

received  at  a  particular  run  and  must  be  calculated  from  the  equations 
(4.32)  through  (4.34)  on-line.  Furthermore,  even  for  a  time-invariant 
system,  this  estimator  does  not  possess  a  steady  state  structure  due  to 
the  fact  that  the  solution  of  equations  (4.  32)  through  (4.  34)  does  not  converge 
to  a  steady  state  as  time  increanes. 

These  disadvantages  are  avoided  in  the  estimator  we  now  derive. 

The  approach  is  again  to  bound  the  set  of  possible  states  consistent  with 
the  observations  by  an  ellipsoid.  In  contrast  to  (SI),  we  do  this  indirectly 
by  bounding  the  instantaneous  constraints  (4.30)  with  an  energy  constraint  of 
the  form  (4.3)  and  then  using  the  results  of  Section  4  to  produce  an  ellipsoidal 
bound  on  We  will  restrict  our  attention  to  the  filtering  problem.  En¬ 

tirely  similar  arguments  can  be  used  to  derive  estimators  for  the  prediction 
and  smoothing  problems. 

An  energy  bound  for  the  instantaneous  constraints  (4.  30)  is  given  in 
the  following  lemma: 


-1 


i+1  | i+l^i+l^i+1  depends  on  the  measurements 


Lemma  4.2:  The  eet  U.  C  Rn+k<r+P>  where 
-  k 


Uk  =  {x0.w0...,Vi, vklx^‘x0  <  1.  <  1. 

VJR^V.  <  1.  i  =  1,2,  ...k} 


li  l  — 
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is  contained  in  the  set 

♦  1^1 

Uk  =  (W'VpV-’klW  *o+  ?<*2.i-lwl-/1i.lwM 

1=  1 

+  a3,  iViRi  lyi  -  (4-39) 

where  a^,  a^.  a^  .,  i=  l,2,..,k,  are  any  nonnegative  real  numbers 
with 


*1  +  +  =  1 


(4.40) 


Proof:  Multiply  (4.  30a,  b,  c)  by  a^ ,  a3  i*  respectively,  sum  the 

last  two  from  i  =  1  to  i  =  k  and  use  (4. 40). Q.  E.  D. 


Having  bounded  the  instantaneous  constraints  (4.38)  by  the  energy 
constraint  (4.  39),  we  are  now  in  a  position  to  apply  the  results  of  Proposi¬ 
tion  4.2  to  give  a  bounding  ellipsoid  to  the  set  X^|  The  equations  that 
result  by  application  of  Proposition  4.2  become  simpler  if  we  write  a.,  a. 

\  L% 

and  .  in  the  following  form: 

aj  =  d-po)(i-p1)(i-p1)(i-p2) _ (i"Pk-j)<i-Pk> 

*2,0  =  Po0-P1)U-Pi)0-P2)---(l-Pk-l)<1-^ 

*3,1  =  P1(l-P1)d-P2)---(1-Pk.1)(l-Pk)  (4.41) 


a2,k-l  =  Pk-l(1"pk) 

*3,  k  ~  ^k 

where  p .  j ,  p  i  =  1 ,  2,  .  . ,  k  are  any  real  numbers  with  0  <  p^j  <1,  0  <  p.  <  1 . 
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It  is  easy  to  see  that  for  the  parameters  aj»  a2  j  a3  •  as  defined 

k  *  ’ 

by  equations  (4.41)  we  have  a.  +  2  (a,  .  ,  +  a~  .)  =  1. 

1  i=  1  1 

By  combining  now  Lemma  4.2  under  the  identifications  (4.41)  with 
Proposition  4.2  we  have  after  straightforward  manipulation  the  following 
solution  to  Problem  4.2  for  the  filtering  case. 


Proposition  4.3:  A  bounding  ellipsoid  to  the  set  of  system  states  X^j  k  of 
Problem  4.2  is  given  for  all  k,  0  <  k  <  N,  by  the  equation 


xJlt  =  {x|(x-^k)»r"jk(x-xk)  <  1  -  6Z(k)> 


(4.42) 


where  the  positive  definite  symmetric  matrix  £k|  k  is  given  recursively  by 
the  equations 


xili  ■  l(1‘pi,zi:|Ii-i  +pic!Ri:*ci1"1 


(4.43) 


ziU-i  =  (1  |  t-iAl-i  +pi"-iBi-iQi-iB!-i  <4-44> 


o  o 


(4.45) 


the  vector  xk  is  the  solution  of  the  equation 

=  A.x.  +  |  •  .iC1.  ,,R7^,(z.  , ,  -C...A.&.) 

i+l  i  i  Ki+1  l+l  l+l  i+i  i+l'  l+l  l+l  i  i 


(4.46) 


with  the  initial  condition 


x  =  0 
o 


(4.47) 


and  the  nonnegative  scalar  6  (k)  is  the  solution  of  the  equation 
62(i)  =  (l-p._1)(l-p.)62(i- 1) 

+  (z.-CA.  | x .  1),[(l-pi)"1C.r.|.  ,C!  +  p^R.l'^z.-CA  ,x.  .) 
'  l  l  l-l  i-l'  1  ri  i  i|i-l  i  ri  'i  i  »-l  i-l' 


(4.48) 
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with  the  initial  condition 

62(0)  =  0  (4.49) 

and  ,  p.,  i  =  1, 2, .  . ,  N,  are  any  real  numbers  with  0  <  (1.  ^  <  1, 

0  <  p.  <  1. 

r  i 

It  can  be  seen  that  the  estimator  of  the  above  proposition  has  a  similar 
structure  with  the  stochastic  '  Kalman  filter  as  well  as  with  the  estimator 
of  Lemma  4. 1 .  However,  it  has  the  important  advantage  over  the  latter  that 
the  gain  matrix  {pi+jZ.+j  j  .+jC!+jR.^j}  is  precomputable  once  the  parameters 
p.  j ,  p  are  selected.  Furthermore,  as  will  be  discussed  in  the  next  section, 
for  a  time -invariant  system  the  estimator  of  Proposition  4.3  can  be  imple¬ 
mented  as  a  time -invariant  system  if  the  final  time  N  approaches  infinity. 

In  practical  applications  this  last  advantage  can  be  of  extreme  importance. 

A  vital  question  concerns  the  comparison  of  the  quality  of  approxi¬ 
mation  to  the  set  of  possible  states  provided  by  the  two  estimators.  It  turns 
out  that  the  approximation  is  comparable  in  the  following  sense.  Let 
{p;,  p*j,  .  .  .Pn.j.pJnj}  be  a  set  of  parameters  used  in  the  estimator  of  Lemma 

4.  1  and  ^PqiPj»  •  •  •  '  Pn-I’^N^  a  8et  Pararrieter8  U8®d  in  the  estimator  of 

$ 

Proposition  4.3.  Then  if  we  select  for  i  =  1, 2,  .  . ,  N 


P! 


P; 


i-1 


i_1  (l-62(i-l)](l-Pi_1)+P._1 


P; 


{(i-6T(i-i))(i-Pi.1)  +p._1}(i-pi)  +p. 


(4.  50) 


(4.51) 


This  fact  was  brought  to  the  author's  attention  by  F.  Schlaepfer. 


k. 
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2 

where  6  (i-1)  is  the  measurement  dependent  term  of  equation  (4.48)  in 

* 

Proposition  4.  3,  the  estimate  ellipsoids  provided  by  the  two  esti¬ 

mators  are  identical  for  all  k  and  for  all  sets  of  received  measurements 
for  which  equations  (4.  50)  and  (4.  51)  hold. 

Another  important  question  concerns  the  quality  of  the  approxi- 
mation  of  the  bounding  ellipsoid  produced  by  the  estimator  of  Propo¬ 
sition  4.3  to  the  exact  set  of  possible  states  This  is  a  question 

largely  unresolved  to  this  date.  It  appears  to  be  very  difficult  to  obtain 
estimates  of  the  approximation  involved  which  will  be  applicable  to  a  large 
class  of  problems.  For  any  given  problem  however  to  is  possible  to  esti¬ 
mate  exactly  the  approximation  in  any  direction  as  it  will  be  discussed  in 
Section  8.  A  related  problem  which  will  also  be  discussed  in  Section  8  is  the 
question  of  the  optimal  selection  of  the  parameters  Pi _j  and  p . . 


7.  Constant  Systems  and  Infinite  Time  Intervals 


In  this  section  we  consider  the  special  case  of  Problem  4.2  where 
the  system  and  the  disturbance  ellipsoid  sets  are  constant,  i.e.,  =  A, 

=  B,  =  C,  =  Q,  R^  =  R  for  all  k.  If  we  select  the  parameters  p^, 
p^  to  be  also  constant  (i.e.,  P^  =  P,  p^  =  p  for  all  k),  the  equations  (4.43), 
(4.44)  for  the  matrix  2^^  in  Proposition  4.  3  become 


Aik  ■  [<i-P>A}k.1+‘’c'*'1cr1 

(4.52) 

A|k-i  3  o-er’A^.ii^A'+p-W 

(4.53) 

with  initial  condition  Lq|o  =¥  .  These  equations  can  be  put  into  the  usual 
discrete -time  Riccati  equation  form 
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sk|k  *  (skjk-i  +  c,R*'lc>'1  <4-54> 

V-i  *  AVi|  k.iA*' + bq*b'  <4-55> 

l(l  jb  A 

by  defining  the  matrices  A  ,  Q  ,  R  as 

A*  =  (l-|3f  1/2(l-p)'1/2A,  Q*  =  P'^l-p)'^,  R*=p-1R  (4.56) 

(T1 ) 

It  is  well  known'  '  that  the  solution  ^  of  equations  (4.  54),  (4.  55)  con¬ 
verges  to  a  positive  definite  symmetric  matrix  2^  as  k  -*■  oo  if  the  pair 

*  * 

(A  ,  C)  is  completely  observable  and  the  pair  (A  ,  B)  is  completely  control- 

lable.  The  pair  (A  ,  B)  is  completely  controllable  if  and  only  if  the  pair 
(A,  B)  is  completely  controllable  i.  e. ,  the  constant  system  (4.1)  is  com¬ 
pletely  controllable.  This  can  be  seen  by  the  fact  that  the  matrix  A*  is  a 
scalar  multiple  of  the  matrix  A  and  therefore  the  subspace  spanned  by  the 
column  vectors  of  the  matrix  AmB  is  the  same  as  the  subspace  spanned  by 
the  column  vectors  of  the  matrix  A*mB  for  all  m  =  0, 1,  ... ,  n-1.  Similarly, 
the  pair  ( A *,  C)  is  completely  observable  if  and  only  if  the  pair  (A,  C)  is  com¬ 
pletely  observable.  Thus,  for  a  completely  controllable  and  observable 
time -invariant  system,  the  gain  (2^  ^CR*"*}  in  the  estimator  of  Proposi¬ 
tion  4.3  after  an  initial  transient  will  converge  to  a  steady  state  constant 
gain  {Z^C'R  }.  For  practical  reasons,  one  would  like  to  implement  the 
estimator  as  a  time -invariant  system  using,  the  steady-state  gain  for  the 
whole  time  interval,  i.e.,  starting  at  the  initial  time  k  =  0.  This  is  possible 
since,  as  we  will  prove  below,  the  approximation  that  results  by  neglecting 
the  initial  transient  vanishes  as  time  goes  to  infinity. 

Using  the  identifications  (4.  56),  the  estimator  of  Proposition  4.3  for 
a  time-invariant  system  gives  the  estimate  ellipsoid 
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Xk|k  *  {ltl(!t-\>'Sk|k(x‘*k>  -  1  -  «2<k»  (4.57) 

where 

=k|k  *  <2fc|fc-l  +c'S*'‘c  I'1  (4.58) 

^Ik-l  =  A*Sk-l|k-lA*  +  BQ*B'  (4.59) 

Jk+1  *  ASk+Zk+l|k+lC,R*'1(xk+l  -CAik»  (4.60) 

«2(k)  *  (l-Pk.i)(l-pk){2(k-l) 

+  K-CA^.jrtcz^jC’  +R*)_1(zk  -  CAik_j)  (4.61) 

with 

Solo  =*  '  *Q  =  °*  «2(°>  =  0  (4.62) 


If  *1c|k  Zoo  and  ^klk-l  ~*%o  as  k  —  00  and  we  implement  the  estimator  as 
a  time -invariant  system  using  the  steady-state  gain  {^CR*'1}  the  resulting 
estimate  ellipsoid  will  be  given  by 


Yklk 

■  W<*-?k>'2£,1(*-?k>  <  1  -  «2(k» 

(4.63) 

where 

^k+1 

=  A^k+^toC,R*'1(zk+l-CA^k> 

(4.64) 

^(k) 

■  d-P)(l-P)6 Vl)  +  (zk-CAyk_j  ((Ce^C'+R*)" 

1(zk-CA^.i) 

(4.65) 

with 

ya  *  °.  ?2(o)  =  o 

(4.66) 

U.ing  the  (act  that  ^  and  z^.,  -  ^  as  k  —  oo,  it  will  now  be 

proved  that  $k  -  xfc  and  ?2(k)  -  62(k)  as  k  ^  oo,  i.e.,  that  the  estimate 
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$ 

ellipsoid  of  equation  (4.63)  "converges"  to  the  set  of  equation 

(4.  57)  as  k  oo.  To  this  end  let  2^|k  =  2^  +  where  -*  0  as  k  -*  oo. 
Then  from  equations  (4.60)  and  (4.64)  we  have 

*k+l  -Vk+1  -  (A-E00C'R*-1CA)($k-?k,  +  HkC.R*-1(2k+1-CA$k)(4.67) 

*-l 

Now  note  that  the  matrix  (A  -  2^0  R  CA)  is  stable  (has  eigenvalues  within 
the  unit  disk),  since  by  equation  (4.  56) 


A  -  EooC,R*”1CA  =  (l-P)1/2(l-p)1/2(A*-EQoC'R*‘1CA*) 
and  the  matrix  (A  -  2^C'R  CA  )  is  stable  by  a  well-known  property  of  the 

A,  1 

Riccati  equation.  Furthermore,  the  driving  term  H^C'R  (z^+j  -  CAx^) 
goes  to  zero  as  k  **  oo  since  "*  0  as  k  “*•  oo  and  (z^+j  -  CAx^)  is  bounded. 
Therefore,  the  solution  of  equation  (4.67)  goes  asymptotically  to  zero  as 
k  ao  and  hence  y^  -*  &k  as  k  -*  oo. 

Also  from  equations  (4.61)  and  (4.65) 


62(k+l)  -  S^k+l)  =  (l-(3)(l-p)[62(k)-  62(k)]  +ek+1  (4.68) 

where 

£k+1  1  <"k+i  -  CASk>'<C2k+i  |  kc' +  R*>"'<*k+i  -  CA$k> 

-  <‘k+l  -  CAV<CV  +  E*>'1<^k+1  -  CA?k> 


Since  y,  -*  x,  and  2.  , ,  . .  -*  2  as  k  —  oo  we  have  e.  -►  0  as  k  —  oo  and 
7k  k  k+1 1  k  oo  k+1 

since  0  <  (l-f3)(l-p)  <  1  the  solution  of  the  equation  (4.68)  goes  to  zero  as 

~2  2 

k  -*  oo.  Hence  6  (k)  —  5  (k)  as  k  -»  oo. 


Thus,  in  applications  where  the  system  is  constant  and  the  final  time 
approaches  infinity,  one  can  use  the  steady-state  time -invariant  estimator 
and  be  assured  that  the  error  that  results  from  neglecting  the  initial  trans¬ 
ient  of  the  solution  of  the  Riccati  equation  vanishes  as  time  increases. 
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8.  Discussion  and  Sources 

Two  state  estimation  problems  which  involve  a  linear  system  and  a 

set-membership  description  of  the  uncertainty  were  examined  in  this 

chapter.  For  the  case  of  an  energy  constraint  on  the  uncertain  quantities  the 

set  of  possible  states  consistent  with  the  measurements  was  shown  to  be  an 

ellipsoid  which  was  characterized  by  recursive  estimators  similar  to 

Kalman  estimators  used  in  stochastic  estimation  problems.  The  results 

for  the  energy  constraint  case  were  then  used  to  obtain  bounding  ellipsoid 

estimators  for  the,  more  often  appearing  in  practice,  c  i*-.  of  instantaneous 

ellipsoidal  constraints  on  the  uncertain  quantises.  Thi  se  estimators  have 

the  same  basic  structure  as  the  Kalman  estimator  and  ~,ff.  r  distinct  ad¬ 
ze  (SZ) 

vantages  over  existing  schemes'  "  '  . 

The  basic  practical  advantage  of  the  estimators  proposed  in  this 
chapter  is  that  they  provide  intelligent  designs  with  a  minimal  amount  of 
information.  Instead  of  requiring  precise  statistics  of  the  uncertain  quan¬ 
tities  only  bounds  on  the  magnitude  or  energy  of  the  uncertain  quantities 
are  necessary.  Since  the  estimators  have  the  same  basic  structure  as 
Kalman  estimators  the  approach  used  here  in  effect  suggests  an  intelligent 
way  of  selecting  the  gain  matrices  of  the  estimator  with  a  minimal  amount 
of  information. 

One  of  the  questions  yet  largely  unresolved  concerns  the  quality  of 
the  approximation  involved  in  the  algorithms  for  the  instantaneous  constraint 
case.  Related  to  this  question  is  the  problem  of  optimal  selection  of  the 
free  parameters  and  p.  that  appear  in  the  algorithms.  There  are  two 
difficulties  related  to  this  problem.  First  a  criterion  for  optimization  must 
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be  chosen.  Second  an  optimization  algorithm  must  be  devised  based  on 
this  criterion.  Even  choosing  a  good  criterion  is  a  difficult  question. 

For  instance  a  method  which  appears  at  first  sight  to  be  reasonable  is  to 
find  the  parameters  p.,p.  for  which  the  trace  of  the  weighting  matrix  ^ 
at  the  final  time  is  minimized.  An  algorithm  for  selection  of  the  parameters 
so  as  to  optimize  this  criterion  was  derived  by  the  author  yet  for  some 
simple  examples  the  resulting  selection  of  the  parameters  led  to  an  indeed 
poor  design.  Presently  there  exists  no  optimization  algorithm  for  selecting 
the  parameters  P^p.,  and  some  trial  and  error  must  be  used  for  their  selec¬ 
tion.  For  the  case  of  a  time -invariant  system  and  an  infinite  time  interval 
this  is  not  very  troublesome  since  in  this  case  only  two  parameters  p,p 
must  be  selected  with  0  <  p  <  1,  0  <  p  <  1. 

Given  now  any  bounding  ellipsoid  estimator  of  the  form  appearing  in 

Proposition  4.  3,  and  any  set  of  measurements  Zj,  ,  z^  a  comparison 

]|( 

of  the  bounding  ellipsoid  X^|  k  with  the  exact  set  of  possible  states  X^j  k  can 
be  made  in  any  direction  x*  by  comparing  the  value  of  the  support  function 


<r(x*jX*|k)  =  <x^k>+ll-6‘(k)]1,VX|kx’I‘) 


2/u\i  1  /2#  *• 


,*vl/2 


with  the  value  of  the  support  function  t(x  |Xji  k).  This  latter  value  can  be 


calculated  from 


<r(x  IX.  I .  )  =  »up  <x  ,x.  > 


‘ktXk|k 


subject  to  the  constraints 


x. , ,  -  A.x.  +  B.w.  , 

i+l  li  l  l  ’ 


i  =  0,  1,  .  .  ,  k-1 


z.  =  C.x.  +  v.  , 

l  ii  l 


i  =  1,  2,  .  . ,  k 
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X 

1 

X 

A 

1 

O 

o  — 

w!  Q.  *w.  < 

1  , 

i  =  0,  1 ,  . .  . ,  k- 1 

l  i 

i  — 

v!  R7Jv.  < 

li  i  — 

1. 

i  =  1, 2, . . ,  k 

a  linear  program  with  linear  and  quadratic  constraints.  A  comparison  of 
these  values  for  a  number  of  directions  of  interest  and  for  a  variety  of  sets 
of  measurements  can  be  informative  concerning  the  quality  of  the  approxi¬ 
mation  of  the  estimates  provided  by  the  given  bounding  ellipsoid  algorithm. 
We  mention  that  the  question  of  parameter  selection  and  of  the  quality  of 
approximation  have  been  discussed  by  Schlapfer!^c^  Some  simulations 
can  also  be  found  in  the  same  reference. 

Similar  results  to  those  obtained  in  this  chapter  can  be  derived  for  a 
variety  of  problems  not  explicitly  considered  here.  One  such  problem  was 
briefly  discusned  in  Section  3  and  concerns  the  case  where  there  is  no  mea¬ 
surement  noise  in  equation  (4.2)  and  the  energy  constraint  is  of  the  form 

-1  N-X  -1 
x'¥  x  +  2  w!Q.  w.  <  1 
o  o  .  =  0  1  1  1  - 

The  estimator  for  this  problem  is  very  similar  to  the  corresponding  sto- 
(Tl) 

chastic  estimator'  and  can  be  used  to  obtain  a  bounding  ellipsoid  algo¬ 
rithm  for  the  related  instantaneous  constraint  case  where  there  is  no  mea¬ 
surement  noise  by  using  a  similar  bounding  operation  to  the  one  in  Section 

6.  Another  problem  that  can  be  treated  similarly  is  the  static  estimation 
(S2) 

problem  which  does  not  involve  a  dynamic  system. 

The  continuous  time  counterparts  of  the  estimators  of  this  chapter 
have  already  appeared  in  (B2).  The  approach  used  in  this  reference  was 
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to  associate  the  estimation  problem  for  the  energy  constraint  case  with 
the  standard  tracking  problem  of  optimal  control  theory  in  which  time  is 
reversed.  This  approach  can  also  be  used  for  most  of  the  problems  con¬ 
sidered  here,  and  has  the  advantage  that  it  demonstrates  in  a  direct  way 
the  duality  oetween  linear  estimation  problems  and  linear  quadratic  optimal 
control  problems.  However  the  approach  used  here  is  more  efficient  in 
that  it  is  applicable  to  more  general  cases.  In  particular  it  is  applicable 
to  those  problems  for  which  the  estimate  ellipsoid  is  degenerate  (has  a 
weighting  matrix  which  is  positive  semidefinite  but  not  positive  definite). 
Furthermore  it  proves  explicitly  the  one -one  correspondence  between 
estimation  problems  with  an  energy  constraint  description  of  the  uncer¬ 
tainty  and  linear  minimum  variance  stochastic  estimation  problem.  The 
reader  familiar  with  the  Hilbert  space  formulation  of  stochastic  estimation 
problems^111  ^  will  have  no  difficulty  observing  from  the  proof  of  Proposi¬ 
tion  4. 1  that  the  solutions  of  Problem  4.  1  and  Problem  4.  1'  involve  dual 
applications  of  the  projection  theorem  which  result  in  identical  equations. 

Estimation  problems  involving  a  set-membership  description  of  the 
uncertainty  were  first  considered  by  Witsenhausen'  '  in  the  framework 
of  minimax  control  problems  with  imperfect  state  information.  The  set 
description  approach  towards  the  estimation  problem  gained  attention  fol- 
lowing  the  work  of  Schweppe'  ”  '  '  who  demonstrated  that  by  using  ellip¬ 

soidal  approximations,  algorithms  with  potential  for  practical  applications 
could  be  devised.  The  results  of  this  chapter  were  in  fact  largely  motivated 
by  Schweppe1  s  work.  Extensions  of  Schweppe1  s  algorithms  to  distributed 
parameter  systems  were  obtained  by  Schlaepfer!^c ^  Such  extensions  should 
be  possible  for  the  results  of  this  chapter  as  well.  An  estimation  problem 
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which  does  not  involve  ellipsoids  is  the  one  which  involves  a  linear  dis crete - 
time  system  and  instantaneous  polyhedral  constraints  for  the  uncertain 
quantities.  Such  constraints  are  interesting  because  the  resulting  set  of 
possible  states  consistent  with  the  measurements  can  be  characterized 
precisely  by  a  finite  set  of  bounding  hyperplanes.  However  the  number  of 
these  bounding  hyperplanes  increases  with  time  thus  possibly  creating  a 
serious  storage  as  well  as  computational  problem.  A  method  for  obtaining 
polyhedral  approximations  to  the  set  of  possible  states  using  only  a  fixed 
number  of  bounding  hyperplanes  is  discussed  by  Hnyilidza.  ' 

Finally  it  should  be  noted  that  the  results  presented  in  this  chapter 
rely  heavily  on  the  linearity  of  the  system,  and  it  appears  to  be  quite  dif¬ 
ficult  to  obtain  extensions  to  nonlinear  estimation  problems.  However  such 
problems  have  not  been  sufficiently  explored  up  to  now  and  are  worthy  of 


attention. 


CHAPTER  5 


MINIMAX  CONTROL  PROBLEMS  WITH 
IMPERFECT  STATE  INFORMATION 

1 .  General  Remarks 

We  now  turn  our  attention  to  minimax  control  problems  with  im¬ 
perfect  state  information.  We  will  consider  the  general  Problem  1.  1 
which  was  introduced  in  Chapter  1.  The  special  case  of  this  problem  where 
the  system  is  linear,  the  cost  functional  has  some  convexity  properties,  and 
the  controller  has  available  an  exact  measurement  of  the  system  state  has 
been  examined  in  Chapter  2.  The  additional  structure  of  this  special  case 
allowed  us  to  obtain  results  that  are  considerably  stronger  than  those  that 
can  be  deduced  for  the  general  Problem  1.1.  For  this  latter  problem  if  is 
very  difficult  to  obtain  results  concerning  existence  of  solutions  or  neces¬ 
sary  conditions  for  optimality.  Furthermore  the  solution  of  the  problem  by 
dynamic  programming,  which  will  be  presented  in  Section  3,  becomes  ex¬ 
tremely  complicated  in  general.  This  is  due  mainly  to  the  fact  that,  as  will 
be  demonstrated,  the  optimal  controller  performs  the  dual  function  of  state 
identification  and  system  actuation.  The  complexities  of  this  situation  are 
well  known  from  stochastic  optimal  control  problems. '  ’  '  We  will  be 

able  to  obtain  insight  into  the  dual  function  of  the  optimal  controller  through 
the  notion  of  a  sufficiently  informative  function  which  parallels  the  familiar 
notion  of  a  sufficient  statistic^*^  of  stochastic  optimal  control.  The  notion 
of  a  sufficiently  informative  function  will  be  introduced  in  Section  4,  and  it 
will  be  used  for  demonstrating  the  separation  of  the  optimal  controller  into 
an  estimator  and  an  actuator.  The  special  case  of  a  linear  system  where 
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the  uncertain  quantities  satisfy  an  energy  constraint  will  be  further  inves¬ 
tigated  in  Section  For  this  case  it  will  be  shown  that  the  estimator  part 
of  the  optimal  controller  can  be  easily  and  efficiently  characterized.  Still 
for  this  case  the  actuator  part  of  the  optimal  controller  cannot  in  general 
be  easily  characterized  although  we  shall  demonstrate  the  characterization 
of  this  actuator  for  the  special  case  of  a  reachability  problem  in  the  next 
chapter. 

In  the  next  section  we  restate  and  briefly  discuss  the  Problem  1.  1 
which  is  the  object  of  study  of  this  chapter. 

2.  Problem  Formulation 

We  shall  consider  the  following  problem: 

Problem  5.1:  Given  is  the  discrete-time  dynamic  system 

r  *k^Xk'  Uk,wk^ 1  k  =  0,  1,  .  .  . ,  N  - 1  (5.1) 

where  x^c  Rn,  k  =  0,  1 ,  .  .  .  ,  N  is  the  state  vector,  u^f  Rm,  k  =  0,  1 ,  .  .  .  ,  N- 1 , 

is  the  control  vector,  w^e  R  ,  k  =  0,  1 ,  .  .  .  ,  N-  1 ,  is  the  input  disturbance 
vector,  and  f^  :  Rn x  Rm x  Rr  — '  Rn  are  known  functions. 

Available  to  the  controller  are  measurements  of  the  form 

zk  =  hk(xk.  ,  k*  0.1 . N-l  (5.2) 

where  z^e  R8,  k  =  1,2,...,  N-l,  is  the  measurement  vector,  v^e  R^,  k  - 

1 , 2,  . .  . ,  N- 1 ,  is  the  measurement  noise  vector,  and  h^  :  Rn  x  R*5  -*  R  8 

are  known  functions. 

.  .  ....  ,  ,  .  .  „ n+Nr+(N - 1  )p 

The  uncertain  quantities  lumped  in  a  vector  qc  R  r 

9  ~  ^X£j*  Wy  ’  w'i  *  •  •  ’  •  _  j  >  v'j »  v->  i  •  •  •  i  | 


(5.3) 
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are 


•  .  ,  ,  .  ,  .  _  .  _n+Nr+(N-l)p 

known  to  belong  to  a  given  subset  Q  of  R 


qe  Q 


(5.4) 


Attention  is  restricted  to  control  laws  of  the  form 

k(s+m)  _  Rm,  k  =  0,  1 ,  . . ,  N- 1 


(5.5) 


taking  values 


Uk  =  ^k^Z  1  ’  Z2*  *  ‘  ’  zk' Uo' U1 '  ‘  *  Uk  1^*  ^  =  0*1***»N-1 


(5.6) 


where  is  interpreted  as  a  constant  vector. 

It  is  required  to  find  (if  it  exists)  the  control  law  in  this  class  for 
which  the  cost  functional 


J (f^j*  •  •  *  i  f*N_  | )  =  sup  F(  x  j ,  x^t  •  •  i  xj»j»  z j » u0 ) »  •  •  •  •  •  >ui\j-l^  1 

qe  Q 

(5.7) 


is  minimized  subject  to  the  system  equation  constraints  (5.  1),  and  where 
F  :  pN(n+m)  —  (-oo,  +ao)  is  a  given  function. 


As  in  Problem  2.1,  the  use  of  the  (semiclosed)  extended  real  line  as 
the  range  of  the  function  F  permits  the  incorporation  of  state  and  control 
constraints  in  the  cost  functional  by  adding  to  the  function  F  the  indicator 
functions  of  the  state  and  control  constraint  sets. 

In  the  next  section  we  will  present  a  dynamic  programming  algorithm 
for  solution  of  the  Problem  5.  1.  Using  this  algorithm  we  will  then  be  able 
to  reach  some  conclusions  concerning  the  structure  of  the  optimal  control 


law. 


-99- 


3.  Solution  by  Dynamic  Programming 

Consider  the  optimal  value  of  the  cost  function  (5.7) 

J  =  inf  sup  F(x1(x2,  .  .,xN,  u  .Uj,  .  .  ,uN  j)  (5.8) 

•*k  ’eQ 

k  =  0,  1, .  .,N-1 

The  purpose  of  the  dynamic  programming  algorithm  is  to  convert  the  mini¬ 
mization  problem  indicated  in  the  above  equation  to  a  sequence  of  simpler 
minimization  problems  by  taking  advantage  of  the  sequential  evolution  of  the 
system  state,  and  the  information  available  to  the  controller  according  to 
equations  (5.1)  and  (5.2).  However  matters  are  somewhat  complicated  in 
the  above  problem  by  the  presence  of  uncertainty  since  in  the  process  of 
generating  the  state  and  measurement  vectors  the  disturbances  are  inter¬ 
mediately  selected  by,  say,  Nature  with  the  objective  to  maximize  the  value 
of  the  cost.  For  this  reason  the  development  of  the  dynamic  programming 
algorithm  will  require  a  somewhat  elaborate  construction.  We  give  first 
the  following  preliminary  definitions. 

Let  P(R8)  be  the  power  set  (set  of  all  subsets)  of  R8  and  consider 
the  following  function 

Zj:Rm-P(RP) 

m  ^  n 

which  assigns  to  the  vector  ujtR  the  set  Zj(uq)C  R^  consisting  of  all 
possible  measurement  vectors  Zj  given  by  equation  (5.2)  which  are  con¬ 
sistent  with  the  constraint  set  Q,  the  system  equation  (5.  1)  and  the  control 

A 

vector  u  .  In  other  words  we  have  z.cZ.(u  )  if  and  only  if  there  exists  a 
o  1  1'  o  ' 

vector  q  =  (x^,  w'q,  w'j,  ..,  w^_j ,  v'j ,  v^,  ...  v|s}_  1  )'eQ  such  that 

Z1  =  WV  VWo)tVl] 
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Similarly  we  define  for  k  =  2,  3,  .  . ,  N-l  the  function 


Zk:R(l“1,,+km  -  P(R’) 


(5-9) 


which  assigns  to  the  vectors  z^ ,  z2>  . . . ,  z^  j,  u  ,  ,  . . ,  j  the  set 

A  g 

Z^(z  j,  z2,  .  . . ,  z^  j,u,Uj,..,u^_j)cR  of  all  measurement  vectors  z^  given 
by  equation  (5.2)  which  are  consistent  with  the  constraint  set  Q,  the  previous 
measurement  vectors  j,  and  the  previous  control  vectors 

W  •  ,uk-r 

We  also  define  the  function 


q  ;  r(N-!  )s+Nm  _  p^Rn+Nr+(N-l)pj 


(5.10) 


which  assigns  to  the  vectors  Zj ,  z^,  .  .  . ,  z^  j,  u^,  u^,  . . ,  u^  j  the  set 
Q(zj,z2>  .  .  ,  zN1,uo,u1,  .  .,uN  l)cR  of  all  vectors  q  = 

(Xo’  wo’  w\  *  •  *  •  WN-  i'  *1*  v2*  ’  '  ’  VN-1^  w^ic^  bel°ng  to  tbe  Q  and  are  con¬ 
sistent  with  the  measurements  Zj,  z2,  .  . ,  and  tbe  control  vectors  uq, 

Uj . UN-1*  In  other  words  a  vector  q  =  (x^,  w^.wj,  .  •  *  wn_  j  » v\  •  vz>  • 

belongs  to  the  set  Q(z  j ,  z2,  .  . . ,  T  Uo’  uj*  •  •  •  *  UN-1^  if  atld  on^y  Q  and 

the  vecto  rs  ‘  * 1  *N-l’  '  '  '.'^N-l'Zl'  ****^N-1'  ^o  *  '  *  *  *  - 1 

satisfy  the  system  and  measurement  equation  (5.  1)  and  (5.2)  for  all  k. 

In  order  to  simplify  the  notation  we  will  make  use  of  the  following 

vector  C  ^  k  =  1,2 . N-l,  which  consists  of  all  the  information  available 

to  the  controller  at  time  k 

*2’  ' ' '  zk'  u0  (5.11) 

A  A 

Using  this  notation  we  write  for  the  control  law  and  the  functions  Z^»Q 
of  equations  (5.9)  and  (5.  10) 
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jUj*  z2»  *  '  *  zk*  uo*  U1  ’  *  *  ’  uk- 1  ^  =  Pjj( ^  = 


(5.12) 


Zk(zrz2 . zk-ruo,ur  *,,uk-i)  =  zk(^k-ruk-i)  {5,13) 

A  A  fc 

Q(zj,  z2,  .  . ,  zn_i»  u0*  ui*  *  •  »UN-1^  =  ^  * N-  1*  UN-  1  ^  (5.14) 

It  should  be  noted  that  for  some  vectors  £k  j  it  is  possible  that  the 

A  k  A  fc  jyj 

set  Zk(b  k  j )  or  the  set  Q(  b  ^ ^ ,  UN-1^  is  emPty  *or  uk  ie  R 

implying  that  the  vector  £  k_  j  =  (z'j,  z^,  . . ,  u^,  u'j,  .  .  ,  ^ )'  is  incon¬ 

sistent  with  the  constraint  set  Q  and  the  system  and  measurement  equations. 
Notice  also  that  whether  the  set  Z^(  £k  i,uk  l )  *s  emPty  or  nonempty  depends 
on  the  vector  £  ^  j  alone  ar.d  is  entirely  independent  of  u^  j.  In  equations  to 
follow  in  which  empty  sets  appear  we  will  adopt  the  convention  stated  in 
Appendix  I  that  the  supremum  of  the  empty  set  is  -oo  (sup  <£  =  ~oo).  Another 
possible  approach  would  be  to  restrict  the  domain  of  definition  of  the  functions 

A  A  k  A  k 

Z^,  Q  to  include  only  those  vectors  b  k  j  for  which  the  sets  Zk(bk  j), 

Q(Cn_i»uN-1^  are  nonemPty*  Since  in  any  actual  operation  of  the  system  these 
sets  will  always  be  nonempty  this  restriction  results  in  no  loss  of  generality. 

We  are  now  ready  to  state  and  prove  the  following  dynamic  program¬ 
ming  algorithm. 


Proposition  5.1:  Assume  that  for  the  functions  defined  below  we  have 
-oo  <  Hr(  Ck),  k  =  1, 2,  .  . ,  N-2,  for  all  vectors  £  t  such  that  the  set  Z,^,  ( £u,  uu) 


k+lv*k’“k' 


is  nonempty  (for  all  ukc  Rm),  and  -oo  <  H^_  ^  (£  j )  for  all  vectors  £N  ^  such 

A  k  _ 

that  the  set  Q(b^  i,uN  j)  *8  nonemPty*  Then  the  optimal  value  J  of  the  cost 
functional  (5.7)  is  given  by 

J  =  inf  E.  (u  ) 

1  o 


(5.15) 
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where  the  function  Ej  :  Rm  -*•  (-oo,  +oo]  is  given  by  the  last  step  of  the 
recursive  algorithm 

EN^N-ruN-P  =  A,rSUp  F(xj»  x2»  *  •  •  xN,u0,ul*  *',uN-l) 


qeQ(CN.ruN.i) 


(5.16) 


Hk(£k)  =  inf  Ek+1(Ck,uk),  k  =  1, 2,  . . ,  N-l 

uk 


(5.17) 


Ek+I(tk’V  = 


Z  «T  it  »  ,Hk«({k'V*k+l> 

zk+ieZk+i'sk,V 


f  .  Hk+1 '  *  k+1 


H^,(S.X1),  k=  0,  l,..,N-2 


(5.18) 


zk+leZk+l'Vuk 


Proof:  Consider  the  coat  functional  (5.7) 

J (Hq*  M'j  i  •  • » | )  =  sup  F[  Xj ,  X2 « •  •  xpq»  (^  j )»  •  • »  P jvj _  j  1 )] 

qcQ 

(5.7) 

and  the  functions 


JN-l^o’ Pr  '  '  ’  ^N-2^  =  inf  '  "^N-l1 


(5.19) 


J  —  l)  =  ^k+1  ^1^2 1  '  ' 1  ^  -  1  >  2,  .  . ,  N-2 

Hk  (5.20) 

We  have  for  the  optimal  value  of  the  cost  functional 


J  =  inf  Jj(po)  =  infJj(Uo) 


(5.21) 


To  prove  the  proposition  we  will  recursively  show  the  equations 


JN-i^o,Kr •  ”pn-2)  =  &up  ;••• 


Z1*W  zN-leZN-l^N-2,uN-2* 


.HN-1(^N-1* 


(5.22) 
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*  * '  *  ^k- 1  ^  ~  8!iP  •••• 


sup  ....  sup 

lcZlW  *kcZk^k-l,uk-l) 


Hk(Ck)  (5.23) 


JiK]  =  Ji(uo)  =  Hi(*i*uo)  =  w 

ziezi(V 


(5.24) 


where  in  the  above  equations  uk  denotes,  for  all  k,  the  value  of  the  function 
Pk  at  the  point  £  k> 

The  equation  (5. 15)  which  is  to  proved  follows  then  by  comparing 
equations  (5.21)  and  (5.24). 

We  begin  by  proving  equation  (5.22).  Consider  the  function  of 
equation  (5.  16) 


EN^N-1,UN-1^=  A,}Up  F(Xj*  x2*  •  • »  XN’  Uo*  U1  ’  *  *  ’  uN-i 

qcQ(*N- 1  *  uN- 1 


) 


We  have  EN(  for  all  vectors  £  NJ  such  that  Q( 

=  ^  (for  alluN1€Rm)  and  we  have  ^N(  £N_  j ,  uN_  j )  >  -go  otherwise  since  the 
function  F  does  not  take  the  value  -oo.  By  the  assumption  that  HN  ^(£N  j)  > 
-oo  for  all  £n_j  for  which  Q(  £n-1'uN-1^  ^  ^  we  have 

Hj^_j(£jij_j)  =  inf  Ej^(  ^  n  - 1  ’  UN- 1  ^  =  ”®0’  ^N-l  8UC^  that 


N-l 


^N-l,uN-l}  =  ^ 

Hn_  i  (  ^  N- 1 )  =  EN<  N  - 1  ’  UN- 1 )  ^  ”® »  for  all  £jyj_j  such  that 

UN- 1 

^N-1,UN-1^  ^  ^ 


Thus  for  every  c  >  0  there  exists  a  function 
1  Rm  such  that 
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EN^N-1*  **N-1,€^N-1^  -  inf  EN^N-1,|AN-1^N-1^  + 

^N-l 

=  inf  EN(CN_ltuN_j)  +  c  =  Hn_1(Cn_1)+c 


(5.25) 


N-l 


for  all  eN_lfR(N-1)(8+m) 


We  have  now  from  equation  (5.22)  for  any  fixed  functions 
^N-2  Wlth  Uo  =  *VU1  =  Kl^l**  *  *  *  ,UN-2  =  ^N-2^N-2* 

1  ^o’  ^1 '  *  ’  *  2^  =  *ni  ®UP  ®*^xl *  u2*  ’  *  *  XN*  Uo’  U1  *  *  *  *  UN- 1  ^ 

^n-!  qfQ 


inf 

^N- 


I  JtP  E(X1’U2’  *  *  ,xN,u<y  ul’  •  *  ,UN-1 
1  zlcZl*Uo*  zN-lcZN-l*»N-2,uN-2*  *®«N-1*N-1) 


inf  sjjp 

■‘n-i 

<  sup 
Bie2i 

(by  using  relation  (5.25)) 


^N^N-l^N-l^N-l^ 


mi  sup  ....  sup 

^N-l  zleZl*V  zN-l€ZN-l*»N-2,UN-2* 

BXP,  £up  ,r  EN^N-rpN-l,e(^N-l^ 

zle2l*Uo*  zN-lfZN-l^*N-2’  uN-2 


-  i  v  ’  '  ’  |UP  it  \  ^  EN(SN-l,pN-l^N-l>l  +  e 

zlcZl(uo}  ZN  - lcZN- 1  N-2’  uN-2  ^N-l 


(by  using  the  minimax  inequality) 


inf  sup  ....  bu  p  EN^N-1*^N-I^N-1^  +  c 

^N-l  zlcZl(V  *N-ltZN-l^*N-2,v^l-2^ 

JN-l^o'^r  '  '  *’^N-2^  +  € 


Since  these  relations  hold  for  any  e  >  Owe  have  equality  throughout 
the  above  algebra  proving  equation  (5.22) 

■^-l^o’V  ‘  ’  ,HN-2^  =  8£P  A  sup£  .  inf  EN^N-1»hTsI-1^^N-1^ 

YtV  zN-1€ZN-1^N-2’^M-2^MN-1 
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/#P/  r  “  “N-I^N-I* 

zlc2l(uo)  *N-1C^N-1^  *N-2,UN-2 


By  using  now  equation  (5.22)  the  equation  (5.23)  can  be  proved  recursively 
for  all  k  by  using  identical  arguments  as  the  ones  used  above  to  prove 
equation  (5.22).  The  conclusion  of  the  proposition  then  follows  from 
equations ((5L 21)  and  (5.24).  Q.E.D. 

We  remark  that  the  value  H^(  C^)  of  the  function  of  equation  (5.  17) 

has  the  interpretation  of  the  "cost-to-go"  at  time  k  from  the  point  of  view  of 

the  controller  on  the  basis  of  the  current  information  vector  C,  .  If  the 

k 

vector  consistent  with  the  constraint  set  Q  and  the  system  and  mea¬ 

surement  equations  (as  it  will  always  be  in  any  actual  operation  of  the  system) 
the  value  H^(  C^)  is  either  a  real  number  or  +oo.  In  the  case  where  the  set 
Q  is  bounded  a  value  of  +oo  indicates  similarly  to  the  related  case  discussed 
in  Chapter  2  that  there  exists  a  disturbance  selection  policy  (on  the  part  of 
Nature)  that  can  cause  a  violation  of  a  state  constraint  regardless  of  the 
control  law  that  the  controller  uses  subject  to  the  control  constraints  that 
the  extended  real  valued  function  F  of  the  mst  functional  (5.7)  implies. 

The  optimal  control  law  if  it  exists  can  be  obtained  from  the  algorithm 
as 

j*k(  =  uk*  k  =  0,1 . N-l 

where  u^  is  the  point  where  the  infimum  is  attained  in  equation  (5. 17)  for 
the  fixed  point 

The  dynamic  programming  algorithm  of  Proposition  5. 1  can  be 
profitably  interpreted  in  terms  of  game  theory,  and  in  particular  in  terms 
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of  multistage  games  of  perfect  information.  ^  The  optimal  value  of  the 
cost  J  can  be  viewed  as  the  upper  value  (or  min-max)  of  a  game  played  by 
two  opponents,  the  controller  selecting  the  control  law  po,p.j,  .  . ,  j  , 
and  Nature  selecting  the  uncertain  quantities  q  from  the  set  Q.  The  infor¬ 
mation  based  on  which  the  decision  of  the  controller  is  made,  is  fixed  by 
the  form  of  the  functions  p^,  i.e.,  by  the  information  vectors  Since 

however  only  the  upper  value  of  the  game  is  of  interest  here  a  variety  of 
equivalent  methods  of  selections  of  the  vector  q  and  corresponding  infor¬ 
mation  patterns  can  be  assigned  to  Nature.  One  such  information  pattern 
and  method  for  selection  of  the  components  of  the  vector  q  corresponds  to 
the  following  sequence  of  events 

(1)  Controller  selects  u 

'  o 

(2)  Nature  selects  z,  from  the  set  Z.(u  ) 

1  l  o 

(3)  Controller  selects  Uj 

(4)  Nature  selects  from  the  set  Z^tzj.u^.u^) 

(2N-1)  Controller  selects  j 

(2N)  Nature  selects  all  the  uncertain  quantities  q  = 

(Xq,  wo*  wr  •  •  *  wN-r  v'r  v2*  *  ••  vk-i^' from  the  8et 
Q(  ^N-ruN-P’ 

Each  selection  by  either  Controller  or  Nature  is  made  with  full  knowledge 
of  the  outcomes  of  previous  selections. 

This  sequence  of  events  is  fictitious,  however  it  accurately  reflects 
the  sequence  of  events  as  viewed  by  the  controller  whose  only  information 
concerning  the  course  of  the  game  at  time  k  is  the  information  vector  C^, 
i.  e. ,  all  measurements,  and  all  control  selections  up  to  that  time. 
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A  moment's  reflection  shows  that  in  fact  the  dynamic  programming 
algorithm  determines  the  (pure)  value  J  of  the  game  of  perfect  information 
described  above.  This  value  is  the  same  as  the  optimal  cost  J  of  the  Prob¬ 
lem  5.1. 

Finding  the  optimal  cost  J  and  the  optimal  control  law  from  the  dy¬ 
namic  programming  algorithm  of  Proposition  5.  1  is  in  general  a  very  dif¬ 
ficult  task.  Part  of  the  difficulty  stems  from  the  fact  that,  loosely  speaking, 
the  objective  of  the  controller  is  dual  in  nature;  first  to  actuate  the  system 
in  a  favorable  fashion  and  second  to  try  to  improve  the  quality  of  his  esti¬ 
mate  of  the  uncertainty  in  the  system.  This  is  a  familiar  situation  from 
stochastic  optimal  control,  known  as  dual  control  problem,  the  for¬ 
midable  complexities  of  which  have  been  widely  discussed  in  the  literature. 
In  stochastic  optimal  control  insight  into  the  structure  of  the  optimal  con¬ 
troller,  and  its  dual  function,  can  be  obtained  through  the  notion  of  a  suf¬ 
ficient  statistic!®*^  Similar  insight  will  be  obtained  for  the  minimax  con¬ 
troller  of  this  chapter  by  introducing  in  the  next  section  the  related  concept 
of  a  sufficiently  informative  function. 


Sufficiently  Informative  Functions 


Let  us  consider  the  following  definition: 

Definition  5. 1 :  A  function  S^  :Rk(*+m)  -►  2^  where  is  some  space  will 
be  called  sufficiently  informative  with  respect  to  Problem  5.  1  if  there  exists 
a  function  Ejt+j  ^  x  Rm  [-oo,  +oo]  such  that 


Ek+l*Sk^k),Uk1  =  Ek+l(^k,uk) 


(5.26) 
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where  E^+j  is  the  function  defined  in  equations  (5.  16),  (5.  18)  for  k  =  0, 

1 . N  - 1 . 

The  value  of  a  sufficiently  informative  function  at  any  point  will  be 
called  sufficient  information. 

The  clear  consequence  of  the  above  definition  is  that  if  is  a  suf¬ 
ficiently  informative  function  and  an  optimal  control  law  exists,  then 
this  optimal  control  law  can  be  implemented  as  the  composition 

fki{k>  =  ^'sk(V  <5-27> 


where  p*  is  a  suitable  function  which  can  be  determined  by  minimizing  the 
function  E^+j  of  equation  (5,26)  with  respect  to  u^.  As  a  result  the  control 
at  any  time  need  only  depend  on  the  sufficient  information  S^(  £^),  and 
therefore  if  this  sufficient  information  can  be  more  easily  generated  or 
stored  than  the  information  vector  may  be  advantageous  to  implement 

the  control  law  in  the  form  of  equation  (5.  27). 

Factorizations  of  the  optimal  control  law  into  the  composition  of  two 
functions  as  in  equation  (5,27)  have  been  widely  considered  in  stochastic 
optimal  control  theory,  and  are  commonly  referred  to  as  separation  theo¬ 
rems.  In  such  problems  the  /unction  or  its  value  is  usually  called  a 
sufficient  statistic.  Particularly  simple  sufficient  statistics  have  been 
found  for  problems  involving  a  linear  system,  linear  measurements  and 
Gaussian  white  input  and  measurement  noises.  In  other  problems  suf¬ 

ficient  statistics  of  interest  take  the  form  of  conditional  probability  density 
functions  conditioned  on  the  information  available .  Such  sufficient 

statistics  imply  the  factorization  of  the  optimal  control  law  into  an  esti¬ 
mator  computing  the  conditional  probability  density  function  of  some 


J 
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quantities,  which  may  differ  depending  on  the  problem  given,  and  an  actuator 
Pk  applying  a  control  input  to  the  system.  In  Chapter  4  it  was  demonstrated 
that  in  estimation  problems  which  involve  a  set-membership  description  of 
the  uncertainty  the  set  of  possible  states  consistent  with  the  measurements 
received  plays  a  role  analogous  to  that  of  conditional  probability  density 
functions  in  stochastic  estimation  problems.  Thus  it  should  not  come  as  a 
surprise  that  for  the  Problem  5. 1  we  will  be  able  to  derive  sufficiently  infor¬ 
mative  functions  that  involve  sets  of  possible  system  states  (or  other  quan¬ 
tities)  consistent  with  the  measurements  received.  In  what  follows  we  derive 
such  sufficiently  informative  functions  and  further  concentrate  in  the  well 
behaved  case  of  a  linear  system  and  an  energy  constraint  on  the  uncertain 
quantities  for  which,  as  was  demonstrated  in  Chapter  4,  the  set  of  possible 
states  can  be  characterized  by  a  finite  set  of  numbers.  We  first  introduce 
the  following  notation. 

We  denote  for  all  k  by 

S^(xj ,  •  •  • ,  xk,  wk,  . . . ,  Wj^ _  j ,  Vj^j  *  •  •  •  i  vN _  j  |  (5.  27 ) 

the  subset  of  pkn+(N-k)r+(N  ^  which  consists  of  all  vectors  (xj,  .  .  . ,  x^, 
w^,  .  .  .,w^  vk+i >  •  •  •  *  vn  j)  that  are  consistent  with  the  measurements 

*2 »  *2*  ‘ ’  *k'  control  vectors  uq, Uj . u^  j,  the  system  and  mea¬ 

surement  equations  (5.  1),  (5.2)  and  the  constraint  set  qcQ. 

We  denote  similarly  by 

Sk(xk*  wk’ *  ",WN-r  Vk+1*  ’  * VN-J  (5.28) 


\(xr  x2*  *  *  *  * 


(5.29) 


sk(xlJ  ?k> 
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(5.30) 


the  respective  sets  of  all  possible  quantities  within  the  parentheses  that  are 
consistent  with  the  information  vectoi  the  system  and  measurement 
equations,  and  the  constraint  qeQ. 

With  the  above  notation  we  have  the  following  proposition. 


Proposition  5.2:  A  sufficiently  informative  function  with  respect  to  Problem 
5.  1  is  the  function 

g  #pk(s+m)  ___  p(Rkn+(N-k)r+(N-k-l)p)  x  Rkm 
k 


given  for  all  k  by 

sk(^k)  =  (sk(xi.  •  *»xk'  wk*  •••wN-rvk+r  ,“vN-i^k),uo,ur**,uk-i^ 

(5.31) 

Proof:  Consider  the  function  Ej^  of  equation  (5. 18) 

EN(^N-ruN-l*  =  4,VIp  ,Uj, . 

This  equation  can  also  be  written  as 
EN<  ^N-1,uN-1* 

=  8UP  , ► 

(xi*  x2*  •  • »  XN-1*  wN-l^SN-l(xr  *  •  ,XN-1*  wN-l'  *N-r 


) 


Ffxj.JCj,  *  *'uN-ll 

=  EN<  SN-1^x1’  *  '  ,XN-1,WN-1 1  ^N-l^,uo,ul*  *  ,,UN-2,UN-1^ 

for  a  suitable  function  E^,  proving  that  the  function  of  equation 

(5.31)  is  sufficiently  informative  according  to  Definition  5.1. 
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The  function  Hj^  j  of  equation  (5.  18)  can  now  be  written  as 

HN-1^N-1*  =  inf  ^^-1^’ ••,^M-l,wN-ll^N-l^Uo'ul’ •  •  ,UN-2,UN-1^ 
^-1 

=  nN-l^N-l^T  •  •  ,xN-r  ^N-l^Uo,ul*  *  •  ,uN-2^ 


for  a  suitable  function  H 


N-l 


Now  for  the  function  Ejj  ^  of  equation  (5. 16)  we  have 

EN-1<  ^N-2,uN-2*  =  £  8up,  t  x 

zN-lcZN-l(  *N-2’  uN-2' 

^-l^-l^V  ‘  •»3‘N-rM^-l^N-l^uo,Ul'  ‘  ,,uN-2^ 

(5.32) 

A  ► 

The  set  Z^_j{  bj^  2,V,N  2'  can  described  as 

ZN-l^N-2,uN-2)  =  W-llzN-l  =  ^Vj-l^N-^^N^’  Vn - 2*^*5^ — 2 VN-1^  ’ 

(j^-?y^-2’  vN-l^SN-2^-2’  ^-2’  vN-l^N-2^ 

(5.33) 

where  the  set  Sjnj_2(xjsj_2»  wN-2’  VN- 1  ^N-2^  *8  8et  °*  P°88ikle  vectors 
^XN  2,WN  2,VN  j)  which  are  consistent  with  the  measurements  Zj,  z^,  .  . ,  z^  2 
and  the  control  vectors  uq,Uj,  .  ,,Ujj  ^  according  to  the  system  and  measure¬ 
ment  equations,  and  the  constraint  set  Q. 

The  set  _ 2 (xiq _ 2 *  wN-2*  VN-  1^  ^N-2^  can  0^ta^net^  as  the  projection 
of  the  set  SN2(xj,  .  • .  XN_2'  wN-2*  wN- T  vN-1^N-2^  °n  the  8Pace  of  vectors 
(xN-2*  WN_2'  VN- 1  Therefore  the  equation  (5.33)  can  be  written  in  the  form 

ZN-l^N-2,uN-2*  =  ZN-1^-2^X1*  *  *  *  ^-2’  "fo-2’  "fo-  1*  VN- J  ^N-2),l^l-2^ 

a*  <5'34> 

for  a  suitable  function  j (‘ ,  ' ). 

Also  the  set  S^_j(xj,  . .  ixj^.j»  wn_j  I  ^N_i^  can  he  written  as 
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SN-1^X1*  •  ’  ^N-l^  =  ^xl‘^2*  •  *  *XN-1*  WN-1 1  (5.35) 

XN-1  =  fN-2*xN-2,uN-2’  wN-2^ 

*N- 1  =  hN- 1  ^XN - 1*  VN-1  (xi»  x2*  *  *  *  xN-2*  wN-2*  WN-1  ’  VN-1  ^ 

fSN-2*xl’  ‘  *  *  ,xN-2*  wN-2*  WN-1*  VN-1  I^N-2^ 

=  SN-  1  ^SN-2^X1  *  *  * 1  xN-2'  wN-2*  WN - 1  *  VN -  r  *N-2^’  zN-l,uN-2^ 

* 

for  a  suitable  function  j(» ,  • ,  • ). 

By  substitution  of  equations  (5.34),  (5.35)  in  equation  (5.32)  we 

obtain 


- 1  ^  -  2  *  UN  -  2  ^  =  **  v,  8UP  * 

*  •  ,3^I-2’^-2’ ^-r  VN-1*  *N-2^,11n-2^ 

HN-  1^SN-  l^SN-2(xl'  ’  *  ’  ^J-2*  ^-2'wN-rvN-ll  ^N-2^,ZN-l’UN-2^uo,”,,UN-2^ 

=  EN-1^  SN-2(xl*  ’  ••xN-2’  wN-2’  WN-1*  VN- J  ^N-2*,uo*  Ul’  '  *  ’uN-2^ 


for  a  suitable  function  j,  proving  that  the  function  SN  ^ ^  2^  °*  e<luat*on 
(5.31)  is  sufficiently  informative  according  to  Definition  5.1. 

By  using  identical  arguments  as  above  the  function  S^(  £^)  of  equation 
(5.31)  is  proved  to  be  sufficiently  informative  for  all  k.  Q.E.D. 


From  the  proof  of  the  above  proposition  it  can  be  easily  seen  that  a 
simpler  sufficiently  informative  function  can  bf.  derived  if  some  of  the  states 
x.,  i  =  1,2,..,N  do  not  appear  explicitly  in  the  cost  functional  (5.7).  Thus 
if,  for  example,  the  function  F  ii.  equation  (5.7)  is  of  the  form 

F(x  j »  x2»  ■  •  i  *jsj»  Uq »  1  ’  ’  u0*  ^1’  ’  ’ 1  ^N-l  ^  (5.36) 


then  the  function 
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sk(^k>  =  l^W’*k+i’-'^-i'vk+i-V?-'^-A>-uo>ui . Vi1 

(5.37) 

is  sufficiently  informative. 

Further  simplifications  result  if  the  constraint  set  Q  for  the  un¬ 
certain  quantities  has  a  property  implying  that  the  set  of  values  that  any 
particular  uncertain  quantity  can  take  is  independent  of  the  values  of  the 
other  uncertain  quantities.  We  will  consider  the  case  where  the  set  Q 
has  the  form 


Q  =  {xo»wo,wj,  *  *,wk-l*vl*  v2'  '  *  *  VN  - 1  ^  XoeXo,wieWi*  1  =  °*  **  • 

vkeVk,  k  =  1.2....N-1}  (5.38) 

where  Xq,  W.,  Vk  are  8iven  subsets  of  the  corresponding  Euclidean  spaces. 
The  case  where  the  constraint  Q  is  of  the  form  (5.38)  should  be  considered 
analogous  to  the  case  of  white  input  and  measurement  noises  in  the  corre¬ 
sponding  stochastic  problem.  We  have  the  following  proposition: 


Proposition  5.  3:  Assume  that  the  constraint  set  Q  has  the  form  of  equation 
(5.38).  Then  the  function  Sk  given  for  all  k  by 

^k(  =  [S^(Xj  t  x^,  . . ,  xk  |  ^k)i  ^q»  Uj ,  • .  i  Uj^_  j  ]  (5.39) 

is  sufficiently  informative. 

If  the  function  F  in  equation  (5.7)  has  the  form  of  equation  (5.  36) 
then  the  function  Sk  given  for  all  k  by 

=  ^Sk*xk^k^Uo,Ul’  ”,UN-1^  (5.40) 


is  sufficiently  informative. 
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Proof:  The  proof  follows  by  trivial  modifications  of  the  proof  of  Proposi¬ 
tion  5.2  to  take  into  account  the  special  structure  of  the  set  Q  in  equation 
(5.38).  Q.E.D. 


The  above  propositions  clearly  illustrate  the  dual  function  of  the 
optimal  controller.  By  equation  (5.2 7)  the  optimal  control  law  is  of  the 
form 


(5.41) 


i.e. ,  it  is  the  composition  of  the  sufficiently  informative  function  and  the 

—  >*< 

function  (i^.  The  function  in  the  case  of  Propositions  5.2  and  5.  3  re- 

B  s(c 

presents  an  estimator  and  the  function  represents  the  actuator.  Al¬ 
ternatively  the  optimal  controller  can  be  viewed  as  being  composed  of  two 
cascaded  parts.  The  first  part  produces  an  estimate  set  and  the  second 
part  accepts  as  input  this  estimate  set  and  produces  a  control  vector.  This 
control  vector  is  stored  and  recalled  in  the  future  by  the  controller. 

It  should  be  noted  that  there  is  an  important  difference  between  the 
sufficiently  informative  functions  derived  for  the  Problem  5.  1,  and  suffi¬ 
cient  statistics  for  the  corresponding  stochastic  optimal  control  problem 
in  that  the  possible  additivity  of  the  cost  functional  (5.7)  results  in  no  sim¬ 
plification  for  the  sufficiently  informative  function.  Thus  the  function 
of  equation  (5.  31)  cannot  in  general  be  simplified  if  the  function  F  in  the 
cost  functional  (5.7)  is  of  the  form 

N 

F(xj,  *2*  •  • .  XN.  u0»ui*  •  •  «uN-l)  =  ,Z,  Vxi,ui-1^ 

i=  1 

where  f.  and  g.  ^  are  given  functions.  In  the  corresponding  stochastic  prob¬ 
lem  however  important  simplifications  in  the  sufficient  statistic  result!®*^ 
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This  difference  is  due  to  the  feet  that  where*!  the  expectation  operation 
is  linear  and  distributes  over  addition  the  maximisation  operation  is  not. 

The  equation  ( ft. 4 1 )  which  demonstrates  the  structure  of  the  optimal 
control  law  as  the  composition  of  an  estimator  and  an  actuator  can  provide 
insight  concerning  the  complexity  and  I**  Implementation  of  this  optimal 
control  law.  For  example  as  was  lllustr  *  ed  in  the  previous  chapter  a 
case  where  the  estimator  has  convenient  SiVnii  lure  Is  the  «  aae  of  a  linear 
system  and  an  energy  constraint  for  the  tin*  Main  quan'Mies,  The  suffi¬ 
ciently  informative  functions  of  Proposition  5,4,  I,  l  for  this  case  can  be 
characteriaed  by  a  small  sel  ol  numbers  am  «  an  lie  mmpuled  recursively. 
As  an  illustration  we  state  the  following  pr«  osilton. 


Proposition  5,4:  Consider  the  special  rase  of  Prublem  5,  I  where  the  system 
is  linear 

xk+l  =  Akxk  +  Bkuk  +  Gk#/k*  k  ■  0, 1,,.,N-1  (5.42) 

with  linear  measurements 

zk  =  ckxk+vk*  k  =  1, 2, . . ,  N-l  (5.43) 


and  the  set  Q  for  the  uncertain  quantities  is  specified  by  the  energy  constraint 

(5.44) 


1  N-J  1  N-J  -1 
¥x  +  Sw'.Q.  w.  +  Zv!R.  v.  <1 


°  i=0  ^  1  i=l  11  1  " 


where  ¥,q.,r.  are  positive  definite  symmetric  matrices  for  all  i. 

Assume  also  that  the  cost  functional  is  of  the  form  of  equation  (5.  36y 

F(xj,  *2*  *  *  *  XN*  'o’  ui*  *  *  *  UN-1^  =  ^XN’  u0*  ul  *  *  ’ » UN-1  ^ 

Then  the  function  Sk  :  Rk(s+m)  —  Rn  x  [  0,  1)  x  Rkm  given  by 


(5.36) 
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Sk^kl  =  l*k*62(k)'uo»ul»“»uk.i] 


(5.45) 


is  sufficiently  informative  for  all  k,  where  the  n-vector  x^  and  the  scalar 
2 

6  (k)  are  generated  recursively  by  the  estimator  equations 
0  -  aO  it  r«  p  "  *  _  r  a  &  -  r 


xi+l  "  +  Biui  +  Si+l|i+lCl+lRi+l*zi+l  “  Ci+lAixi‘Ci+lBiui) 


i  =  0,  1,  .  . ,  k- 1 


x  =  0 
o 


ZL|  .  =  [s:,1.  .  +  CIR^C.J  -1 

ii  1  li-l  li  iJ 


(5.46) 

(5.47) 


(5.48) 


2i|i-l  =  Al-lSi-l|i-lAl-l+Gi-lQi-lGi-l 


(5.49) 


2  I  =  ¥ 

o  o 


(5.50) 


62(i+l)  =  6Z(i)  +  ( z ...  -C...A.X.  -C..1B.u.)'(C..12>J,  |  .C| , .  +  R.xl  f1 

'  '  '  l+l  l+l  l  l  l+l  i  i'  '  l+l  l+l  |  i  i+1  i+l' 


(z.,.  -  C.,,A.x.  -  C.  .B.u.) 

'  l+l  l+l  l  l  l+l  i  i 

62(0)  =  0 

Proof:  From  the  results  in  Chapter  4  we  obtain  that  the  set 

sk*xk,wk’ •**wN-rvk+r ••»vn-J  ^k* is  8iven 

Sk(xk’ wk’  ••,wN-l,vk+r*,,vN-J  ^k) 

=  fV  wk-  •  • '  WN.  1  •  Tk+1  •  •  •  •  VN- 1 1  <xk-$k|,Zki  k(xk-*k> 

N-l  .  N-l  .  , 

+  2  w!Q~  w.  +  2  v!R.  v.  <1-6  (k)} 

i=  k .  ’ 1  1  1  i=k+l  1  1  1 


(5.51) 


(5.52) 


(5. 53) 


Since  the  matrix  S^|  ^  is  precomputable  and  the  matrices  and  R^  are  given 
the  set  S^(x^,  w^»  . . ,  wn-1»  vk+l*  *  *  *  VN-1  ^  ^k^  *s  comPletety  determined  from 
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A  2 

the  vector  and  the  scalar  6  (jk).  By  combining  the  equation  (5.37)  and 

the  Definition  5.1  the  result  follows.  Q.E.D. 

Thus  for  the  problem  of  the  above  proposition  the  estimator  part 

of  the  optimal  controller  can  be  completely  and  efficiently  characterized. 

For  the  reachability  case  of  this  problem,  i.e,,  the  case  where 

N- 1 

^XN’  Uo*  ul*  *  *  *  UN-1  ^  =  ^  Uj) 

i=0 

with  X^,  U.  given  sets,  the  actuator  part  of  the  optimal  controller  can  also  be 
completely  characterized  as  we  will  demonstrate  in  the  next  chapter.  Sim¬ 
ilar  results  with  the  Proposition  5.4  can  be  obtained  for  a  general  cost  func¬ 
tional  of  the  form  of  equation  (5.7).  However  in  this  case,  as  can  be  seen 
from  the  Proposition  5.2  and  the  results  in  Chapter  4,  the  sufficiently  infor¬ 
mative  function  will  be  of  the  form 

Sk(V  =  ^l|k’^2|k’ *’,xk|k’6  M’W  *  ""N-l1 

where  x^|^  will  be  given  by  smoothing  equations  for  all  i  <  k. 

For  the  case  of  a  linear  system  with  instantaneous  ellipsoidal  con¬ 
straints  on  the  uncertain  quantities  the  sufficiently  informative  functions  of 
the  Propositions  5.2  and  5.3  cannot  be  characterized  by  a  finite  set  of  numbers 
neither  can  they  be  easily  generated  by  an  estimator  as  was  demonstrated  in 
the  previous  chapter.  This  indicates  that  for  such  problems  the  characteri¬ 
zation  of  the  optimal  control  law  should  be  in  general  very  difficult.  How¬ 
ever  for  the  problem  of  the  reachability  of  a  target  tube  which  involves  such 
constraints  on  the  uncertain  quantities  a  method  for  obtaining  suboptimal 
controllers  that  can  be  more  easily  implemented  will  be  developed  in  the  next 
chapter. 


L 
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5.  Discussion  and  Sources 

The  basic  method  for  the  solution  of  minimax  control  problems 
with  imperfect  state  information  is  the  dynamic  programming  algorithm 
of  Proposition  5. 1.  It  should  be  noted  that  both  in  the  development  and 
the  proof  of  this  algorithm  we  did  not  make  use  of  the  fact  that  the  state 
space,  control  space  and  disturbance  spaces  are  Euclidean  spaces  and  in 
fact  the  Proposition  5. 1  can  be  generalized  for  the  case  where  these  spaces 
are  arbitrary  sets. 

In  general  the  solution  of  the  problem  by  dynamic  programming  is 
a  very  difficult  task,  and  only  for  simple  systems  and  simple  constraint 
sets  such  a  solution  can  be  practical. 

It  appears  that  the  most  well  behaved  special  case  of  Problem  5. 1 
is  the  case  where  the  system  is  linear  and  the  set  of  the  uncertain  quantities 
is  specified  by  an  energy  constraint.  For  this  case  a  sufficiently  informative 
function  can  be  recursively  generated  by  estimators  developed  earlier  in 
Chapter  4.  Even  for  this  case  the  actuator  part  of  the  optimal  controller 
may  not  be  easily  characterized.  In  the  next  chapter  we  will  give  a  pre¬ 
cise  characterization  of  the  actuator  for  the  special  case  of  a  reachability 
problem.  However  it  appears  that  there  is  no  special  case  of  the  Problem 
5.  1  with  a  solution  as  elegant  as  that  provided  by  the  separation  theorem 
of  stochastic  optimal  control  for  linear  systems  and  quadratic  criteria.^^’^^ 

The  only  source  for  minimax  control  problems  with  imperfect  state 

(W1 ) 

information  appears  to  be  the  original  work  of  Witsenhausen;  '  who  dem¬ 
onstrated  the  application  of  dynamic  programming  to  this  problem.  The 
use  of  dynamic  programming  in  sequential  games  has  been  known  at  least 
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since  the  proof  that  finite  games  of  perfect  information  have  a  saddle 
fVl ) 

point.'  The  concept  and  the  development  of  the  Proposition  5. 1  is 
based  on  game  theory  considerations,  and  it  involves  the  construction  of 
a  sequential  game  of  perfect  information  in  its  extensive  form(^u^  The 
algorithm  of  Proposition  5. 1  differs  in  its  form  and  is  more  general  than 
Witsenhausen's  algorithm  however  the  same  basic  ideas  are  involved. 

The  notion  of  a  sufficiently  informative  function  is  introduced  for  the  first 
time  here  in  analogy  with  the  notion  of  a  sufficient  statistic  of  stochastic 
control.  It  has  mainly  theoretical  value  in  that  it  forms  the  basis  for 
demonstrating  the  decomposition  of  the  optimal  controller  into  an  estimator 
and  an  actuator.  This  decomposition  provides  insight  into  the  structure  of 
the  optimal  controller,  and  in  some  cases  it  can  serve  as  a  starting  point 
for  developing  suboptimal  control  schemes. 


CHAPTER  6 


SOME  REACHABILITY  PROBLEMS  WITH  IMPERFECT 
STATE  INFORMATION 


1 .  General  Remarks 

As  was  demonstrated  in  the  previous  chapter  it  is  in  general  very 
difficult  to  characterize  completely  the  optimal  controller  in  minimax 
control  problems  with  imperfect  state  information.  For  the  case  of  a 
linear  system  with  an  energy  constraint  for  the  uncertain  quantities  it 
was  shown,  however,  that  the  optimal  controller  may  be  realized  as  an 
estimator  followed  by  an  actuator  and  that  the  estimator  can  be  easily  and 
efficiently  characterized  using  the  results  of  Chapter  4.  We  shall  dem¬ 
onstrate  in  the  next  section  that  for  the  case  of  the  problem  of  the  reach¬ 
ability  of  the  target  set  the  actuator  part  of  the  optimal  controller  can  also 
be  precisely  characterized,  and  thus  we  shall  give  a  complete  solution  to 
this  problem.  However  the  implementation  of  the  optimal  controller  will 
still  be  quite  difficult  despite  the  simplification  achieved. 

In  Section  3  we  will  consider  a  problem  of  reachability  of  a  target 
tube  which  involves  a  linear  system  and  instantaneous  ellipsoidal  constraints 
for  the  uncertain  quantities.  For  this  problem  it  appears  that,  in  general, 
a  practical  implementation  of  the  optimal  control  law  is  indeed  very  difficult. 
For  this  reason  we  present  a  suboptimal  control  scheme  for  this  problem  by 
making  use  of  the  bounding  ellipsoid  estimation  algorithm  presented  earlier 
in  Chapter  4. 
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2.  Reachability  of  a  Target  Set  for  the  Case  of  Energy  Constraints 

We  first  formulate  the  reachability  problem  that  we  will  consider 
in  this  section. 


Problem  6.1;  Consider  the  linear  discrete-time  dynamic  system 

xk+i  =  ^  ®kuk  ^*kwk  *  k  =  0,  1 ,  .  . ,  N-l  (6.1) 

with  the  linear  measurements 


zk  =  ^-kxk  +vk’  k  =  1,2,..., N-l  (6.2) 

where  x^eRn,  k  =  0,  1,..,N,  is  the  state  vector,  u^eRm,  k  =  0,  1 ,  .  . ,  N-l , 

is  the  control  vector,  w^eRr,  k  =  0,  1,  .  . ,  N-l,  is  the  input  disturbance 

vector,  z^cR8,  k  =  1,2,  .  . ,  N-l,  is  the  measurement  vector,  v^cR^,  k  = 

1,  2,  . . ,  N-l  is  the  measurement  disturbance  vector,  and  Afc,  Bk, 

are  given  matrices  of  appropriate  dimension. 

The  initial  state  x  ,  and  the  disturbances  w.  ,v.  are  assumed  un- 
o  k  k 

known  except  that  they  satisfy  the  energy  constraint 


X  + 

o 


N 

Z  (w[ 
k=l 


■  +vkRklvk)  - 


(6.3) 


where  ¥,Qk  j,Rk>  k  =  1»2,..,N  are  given  positive  definite  matrices. 
Attention  is  restricted  to  control  laws  of  the  form 


Pk:Rk^8+m)  —  Uk,  k  =  0,1,...,  N-l 


taking  values 


U|^  •  fAj^(z  1 1  z 2 1  •  •  9  UQ i  u 2  f  •  • »  U|^  |  )y  k  —  0^  1  ^  |  N  ■  1 
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where  m-q  is  interpreted  as  a  constant  vector  and  where  U^C  Rm,  k  =  0, 

1,  . .  . ,  N-l,  are  given  sets.  It  is  required  to  find  a  control  law  in  this 
class  such  that  the  final  state  of  the  resulting  closed-loop  system  be¬ 
longs  to  a  given  set  Rn  for  all  possible  values  of  the  uncertain  quan¬ 

tities. 


We  will  say  that  the  target  set  is  reachable  if  there  exists  such 
a  control  law. 

The  above  problem  can  be  recognized  as  the  special  case  of  Prob¬ 
lem  5. 1  of  the  previous  chapter  where  the  system  and  measurements  are 
linear,  the  constraint  set  Q  is  specified  by  the  energy  constraint  (6.3)  and 
the  cost  functional  is 

N-l 

=  8UP  l  M*NIXN)  +  £  6(u.|u.)]  (6.4) 

°  1  W  1  qeQ  i=0  1  1 

where  6  (y |  Y)  denotes  the  indicator  function  of  the  set  Y.  Consequently  it 
follows  by  Proposition  5.4  that  for  this  problem  the  optimal  control  law  is 
of  the  form 

^(zj,  z2,  . . ,  zk,uo,Uj,  .  ,,uk  l)  =  P{$c»4lk)*u0'Ur  •  *’uk-l)  f6,  5* 

where  xk  and  6^(k)  are  given  for  all  k  by  the  estimator  equations  (5.46)  through 
(5.  52).  We  will  now  characterize  the  optimal  control  law  (6.  5)  in  the  follow¬ 
ing  proposition. 

Proposition  6.1:  Consider  the  sets  &kc  Rn  x  [  0,  1) ,  k  =  0, 1,  ...  N-l  defined 
recursively  by  the  relations 


-123- 


^N-l  =  «2(n“1)I  3  uN-1€UN-1  such  that 


<an-ixn-i  +bn-iun-i  +gn-iwn-i^xn'  ^xN-r  wN-r 

(XN-1  ”  XN-1  ^'^N-l|N-  1^XN- 1  ■*N-l)+wk-lQN1-l^.li1^<N-1)> 


(6.6) 


=  {*k»6Z(k)|  3  uu€U 


Ic  k 


such  that 


^Akxk  +  Bkuk  +  2tc+l|k+lCk+lRk+ldk^ 

(62(k)  +  dk(ck+12\c+1|kc|c+1  +  Rk+1)  ldk^cXk+r 

Vdk'  dic<ck+i Wck+i  +  Rk+i>'ldk  <  1  -*2<k»  <6*7> 

k  =  0,  1,  . .  .,N-2 


where  the  matrices  2^+1  j  ^+j,  1  ^  are  given  for  all  k  by  the  equations 

(5.48)  through  (5.  50). 

Then  the  target  set  XN  is  reachable  if  and  only  if 

(0.  0)eXQ  (6.8) 

Under  these  circumstances  a  control  law  ^  j }  that  achieves 

reachability  can  be  obtained  as 

uk  =  ^k^zl*  *  "W  *  ,,uk-l*  =  ^k^k’ 6 


where  for  each  pair  ( 


k  =  N-l,  we  have  (AN_JxN_1  +  BN_jUN1  +  GN_1wN_J)€XN  for  all  xN_r 

"N-l  '"lth  *XN-1  ■xN-1*'2'N-1  | N-1^XN-1  +  WN-1QN-1WN-1  -  1  -  S^N- X ), 

if  k  =  0.  1.  . . ,  N-2,  v,e  have  [  ( A ,$k  +  B,uk  +  SR+l  |  k+jC^,®^ ,<y. 

(6  (k)  +  dk(Ck+JZk+l|kCk+l  +  Rk+l,  ldk)'EXk+l  for  aU  dk  wlth 
dk<ck+isk+i|kcUi+Rk+1»'ldk^I-s2<k> 


-124- 


Proof:  We  shall  use  the  dynamic  programming  algorithm  of  Proposition 
5.  1  for  the  cost  functional  (6.4),  and  the  equations  (5.46)  through  (5.  53) 
in  Proposition  5.4.  We  have  from  (5.  17) 


hn-i*Cn-i*  =  inf  A/i-sup  [6*AN-1XN-1  +BN-1uN-1 

UN-1  VQ»N-1*uN-1) 


N-l 


+  gn-iwn-iIxn>  +  =  «<»ilUi)l 


i=  0 


inf  .  BUP  ,  ,  f  ^(AN-IXN-1  *  BN-1uN- 

UN-1  (xN-l’  ^-l^N-l^N-l'^-l^N-l* 


N-l 


+  gn-iWxn>+  .V<uilUi» 

i=0 


Since  by  equation  (5.  53) 

Vl^-r^-l^N-l*  =  ^-l'^-ll^-r^i-P'^N-llN-l^-l  "XN-1) 

+  WN-1QN-1WN-1  <  1  -62(N-1)} 


we  have  that  for  every  such  that  SN-l(xN-T  ^iM-l^N-l^  ^ 


7  A  N-2 

hn_i(Cn.i)  =  «[ftN.r6  (n-dIXn.j!  +  z  «(u.|u.) 


where  j  is  the  set  defined  in  equation  (6.6). 

Using  now  equations  (5.  17),  (5. 1 8)  we  have 

HN-2^N-2^  =  inf  WVl-«2,N-,)l*N.ll 

uN-2  zN-lcZN-l'  *N-2*  UN-2 
N-2 

+  2  6(u  |U.)}  (6.9) 

i=0  1  1 
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for  every  C  ^_2  *uch  that  Z^_j(  C^_2«  uN-2^  *  According  to  the  equations 

(5.46),  (5.  51)  we  write 

XN-1  =  AN-2XN-2  +  BN-2uN-2  +  ^N-l  |N-lCN-lRN-ldN-2  <6*10) 
6Z(N-1)  =  62(N-2)  +  d^_2(C^i.1SN_1|N.2C^_1  +RN-1)  (6.11) 

where 

dN-2  =  ZN-1  '  CN-1AN-2xN-2  '  CN-lBN-2uN-2  (6.12) 

Also  it  can  be  easily  proved  using  the  results  in  Chapter  4  that  the 
set  ^N.1(^N.2*UN_2)  is  8iven  by 

^N-l^N-2,uN-2*  =  *ZN-1  ^N-^^-A-llN^N-l  +  RN-1*  ldN-2-  ^V1"2)) 

where  dN2  is  given  by  (6. 12). 

From  equations  (6.9)  through  (6.  12)  it  follows  that 

HN-2<£N-2>  *  «*N-2-«*<N-*>l*N.21  +N^  6(uilui> 

i=0 

where  ^jq_2  *•  defied  in  equation  (6.7). 

By  proceeding  similarly  we  obtain  that  the  optimal  value  of  the  cost 
functional  (6. 4)  is 

J  =  S[xo,62(0)|Xo] 

2 

Since  we  have  xq  =  0,  6  (0)  =  0  we  obtain 
J  =  o  — >  (0,  0)cX 

o 

J  =  00  <  ...»  (0, 0)^X 

O 

Since  we  have  J  =  0  if  and  only  if  the  target  set  X^  is  reachable  the  condition 
(6.  8)  is  proved. 
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The  fact  that  the  optimal  control  law  ia  of  the  form  indicated  in  the 
proposition  can  be  easily  seen  from  the  preceding  arguments.  Q.E.D. 

A  closer  examination  of  the  Proposition  6. 1  reveals  the  following 
mechanism  for  the  optimal  controller.  First  the  estimator  of  equations 
(5.46) through  (5. 52) 


*k+l  =  Ak*k  +  Bk\  +  \+llk+lCk+lRk+ldk 

(6.13) 

62(k+l>  =  +  +Rk)'Idk 

(6.14) 

dk  =  *k+l  ‘  Ck+1AA  ‘  Ck+lBkUk 

(6.15) 

is  used  to  generate  the  sufficiently  informative  function  which  in  this  case 
is  SjjC^)  =  (  6^(k)]  •  Then  for  any  given  [  6 2(k))  the  controller 

selects  the  control  vector  u^  in  a  way  that  the  "state"  ( $k+j,  62(k+l)]  of  the 
estimator  (6. 13),  (6. 14)  at  the  next  time  instant  (k+1)  will  belong  to  the  set 
&k+1  for  all  possible  values  of  the  error  residual  vector 

dk  =  zk+i  -  ck+iAkik  -  ck+iBkuk 


Thus  in  effect  the  optimal  controller  operates  in  a  way  that  achieves  reach- 

^  2 
ability  of  the  set  Xk+j  by  the  sufficient  information  [*^+1’^  (k+1)]  and 

eventually  reachability  of  the  set  5Cj^_  ^  by  the  sufficient  information 

2  2 

( x^_  j,  64(N-1 )]  .  The  sufficient  information  [  6  (k)]  can  be  viewed  as 

the  state  of  the  (n+1  )-dimensional  system  defined  by  the  equations  (6. 13), 

(6. 14)  with  the  initial  state 

lxo,62(0)]  =  (0,0) 


This  system  is  driven  by  the  control  u^,  and  perturbed  by  the  disturbance 
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vector  of  equation  (6. 15),  and  it  ia  linear  in  the  atate  and  the  control 

but  nonlinear  in  the  diaturbance.  Furthermore  the  diaturbance  d.  aatia- 

k 

fiea  at  each  time  the  conatraint 

di,<ck+i  Vi  I  kck+i  +  Rk+i ‘  dk  i  1 '  I6- 1 6» 

which  ia  a  atate-dependent  conatraint. 

We  can  conclude  from  the  above  diacuaaion  that  in  effect  the  solu- 

tion  of  the  Problem  6. 1  involvea  the  aolution  of  a  target  aet  reachability 

problem  with  perfect  atate  information.  Thia  reachability  problem  however 

doea  not  involve  the  original  ayatem  (6. 1)  but  inatead  it  involvea  the  (n+1)- 

dimenaional  eatimator  deacribed  by  the  equationa  (6. 13),  (6.  14)  the  atate 

A  2 

of  which  ia  the  aufficient  information  [x^.fi  (k)]  .  The  objective  of  the 

controller  ia  to  achieve  reachability  of  the  target  aet  J  by  the  final 

atate  [xN_  j,  62(N-1)]  of  thia  eatimator  aince  if  ( xN1, 64(N-1  ))eXN1  the 

reachability  of  the  target  aet  XN  can  be  guaranteed  by  equation  (6.6).  Since 

the  controller  can  uae  the  eatimator  which  producea  at  each  time  k  the  auf- 

2  . 

ficient  information  1x^,6  (k)]  thia  i8  a  reachability  problem  with  perfect 
atate  information.  However  thia  problem  ia  more  complicated  than  the 
reachability  problema  that  we  conaidered  in  Chapter  3  since  the  disturbance 
d^  of  equation  (6.15)  enter  j  nonlinearly  in  the  equation  (6. 14),  and  the  con¬ 
straint  (6. 16)  is  state  dependent.  For  this  reason  the  construction  of  the 
sets  k  =  0,1,..,  N-l,  is  considerably  more  complicated  than  the  con¬ 
struction  of  the  effective  and  modified  target  sets  that  we  considered  in 
Chapter  3.  As  a  result  the  implementation  of  the  optimal  controller  of 

Proposition  6.1  is  very  difficult  in  general.  By  using  however  internal 

A 

approximations  to  the  sets  X^  it  is  possible  to  derive  suboptimal  control 
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scheme  •  that  achieve  reachability,  and  can  be  more  easily  implemented. 

We  shall  not  pursue  this  matter  here  since  our  primary  objective  in  this 
section  has  been  to  demonstrate  the  interesting  fact  that,  for  the  case  that 
we  consider,  the  problem  of  state  reachability  with  imperfect  Information 
is  equivalent  to  an  estimate  reachability  problem  with  perfect  information. 

3.  Reachability  of  a  Target  Tube  with  Instantaneous 

Ellipsoidal  Constraints 

In  this  section  we  consider  the  natural  extension  of  the  problem  of 
the  reachability  of  a  target  tube  that  was  considered  in  Chapter  3  to  the  case 
where,  instead  of  having  perfect  knowledge  of  the  system  state,  the  controller 
has  access  only  to  noise-corrupted  measurements  of  the  system  output.  We 
will  examine  the  case  of  a  linear  system  and  instantaneous  ellipsoidal  con¬ 
straints  for  the  uncertain  quantities.  As  was  explained  in  the  previous 
chapter  the  implementation  of  the  optimal  controller  for  this  problem  is  in 
general  very  difficult.  Our  objective  in  this  section  will  be  to  develop  sub- 
optimal  control  schemes  that  achieve  reachability  of  the  target  tube,  and 
that  can  be  more  practically  implemented. 

We  will  consider  the  linear  system  of  equation  (6. 1) 

xk+1  =  Akxk  +  Bkuk  +  Gkwfc  ,  k  =  0,  1,  . . ,  N-l  (6.1) 

with  the  linear  measurements 

zk=Ckxk  +  vk’  k=  1,2,..,  N-l  (6.2) 

We  assume  that  the  initial  state  x  and  the  disturbance  vectors  w.  ,  v,  satisfy 

O  K  K 


the  constraints 
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X  Ylx  <  1 

0  o  - 

(6. 17a) 

Ipi'-k  s 1  • 

k  =  0,  1,  ...N-l 

16. 17b) 

vkRklvk  -  1 1 

k  =  1,2,  ...N-l 

(6. 17c) 

where^.Q^,  are  given  positive  definite  matrices. 
We  are  seeking  a  control  law 


=  Rk(<+m)  u, 


k  =  1,2,  . . ,  N- 1 


u|,  =  *2'  *  '  1  '  1  ,UL'.1  ) 


U  =  |i(U 
o  o  o 


k'“o*“r 


‘k-r 


where  U^,  k  =  0,  1,  . .  ,N-1,  are  given  sets,  which  is  such  that  the  state  x^, 
of  the  resulting  closed-loop  system  (6.1)  is  contained  in  the  given  sets  Xk> 
k  =  1, 2,  . .  ,N,  for  each  k,  and  for  all  possible  values  of  the  initial  state  x 

o 

and  the  input  and  measurement  distrubances  w^,  v^  which  satisfy  the  con¬ 
straints  (6. 17). 

We  shall  say  that  the  target  tube  {Xj,  X^,  . . ,  XN)  is  reachable  if 
there  exists  such  a  control  law. 

Given  at  time  k  the  measurements  z z^,  . . ,  z^  and  the  prior  con¬ 
trols  uq,Uj,  .  the  controller  can  calculate  a  bounding  ellipsoid  x£  |k 

to  the  set  X^j  k  of  all  possible  states  x^  consistent  with  these  measurements 
and  controls  by  using  the  bounding  ellipsoid  estimator  of  Proposition  4.3. 
By  taking  into  account  the  presence  of  control  vectors  in  Proposition  4.  3 
we  have  for  all  k: 

.2, 


xk|k  =  {*ki(xk-Vzkik(xk-*k)  -  1  -  >'2<k» 


(6.18) 
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2 

where  6c(k)  ere  given  recureively  by: 

Zi|i  =  I  (1‘pi)Zi|i-l  +  piC!Rilci) 


(6.19) 


Zi|i-1  =  Ai-lZi-l  |i-lAl-l  *  Pi-lBi-lQi-lBl-l  (6.20) 


X  I  V 
•>  o 


(6.21) 


^i+l  '  Ai*i  +  BlUi  +  Pi+l1i+l  |i+lCt+lBi+V'i+l  '  Ci+lAi“i  '  Ci+lBl“i> 


•  0  (6. 

«2(i)  =  (l-Pi.1)(l-Pi)«2(i-n+(ii-qAi.1S1.1.CiBi.1ui.1)' 


(6.22) 

(6.23) 


6‘(0)  =  0 

and  p._  j ,  p^,  i  =  1, 2,  . . ,  N,  are  any  real  numbers  with  0  <  p._j  <  1, 

0  <  p.  <  1. 

Consider  now  the  ellipsoid 

Sk  =  ^XlX,Zk|kX  -  ^  ’  k=  1.2.-..N 


(6.24) 

(6.25) 


(6.26) 


The  ellipsoid  is  precomputable,  since  the  matrix  2^^  does  not  depend  on 

2 

the  measurements, and  expresses  the  maximum  possible  (6  (k)  =  0)  amount 
of  uncertainty  about  the  state  x^  when  the  estimate  x^  is  known.  Since  from 
equations  (6. 18),  (6.  26)  we  have 

Xk|kC  Xk|kCxk  +  Sk 


(6.27) 
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it  i«  clear  that  in  order  for  the  state  to  belong  to  the  set  X^  it  is  suf¬ 
ficient  that  the  state  estimate  belongs  to  the  set 

3k  =  {*k^k  +  SkCXk}  (6.28) 

Thus  for  the  purpose  of  obtaining  sufficient  conditions  for  reachability,  we 
can  shift  emphasis  from  the  problem  of  the  reachability  of  the  target  tube 
{Xj,  X •  • ,  Xj^j}  by  the  system  state  x^»  to  the  problem  of  the  reachability 
of  the  target  tube  {Xj,  X^ ,  ...» by  the  state  estimate  This  latter 
problem  will  be  shown  to  be  a  reachability  problem  with  perfect  state  infor¬ 
mation  of  the  form  that  we  have  already  considered  in  Chapter  3. 

By  substituting  equations  (6.  1),  (6.2)  into  (6.22)  we  have  that  the 
estimate  x^  is  generated  by  the  equation 

xk+l  =  AA  +  Bkuk  +  Lk+ldk  (6.29) 

where  the  lumped  disturbance  d^  is  given  by 

dk  =  Ck+lAk*xk  "  ^k*  +  Ck+lGkwk  +  vk+l  (6,  30* 


and  the  (pre computable)  gain  matrix  is  given  by 

Lk+1  =  Pk+l2V+l|k+lCk+lRk+l 


(6.31) 


Furthermore  it  follows  immediately  from  equation  (6.30)  that  belongs  to 
the  known  set 


Ck+lAkSk  +  Ck+lGkWk 


+  V 


k+1 


(6.32) 


where  is  defined  by  (6.  26)  and  the  ellipsoids  W^, 
the  constraints  (6. 17b)  and  (6.17c) 


are  specified  by 
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wk  •  {wk|wkQk‘wk  i  » 
Vk+1  =  *vk+Jvk+lRk+lvk+l 


<  1} 


Thus  a  sufficient  condition  for  the  reachability  of  the  given  target 
tube  {Xj,  X^,  . . . ,  XN)  by  the  system  state  in  the  presence  of  imperfect 
information  is  that  the  target  tube  . . . ,  XN)  defined  by  (6.28)  be 

reachable  by  the  state  x^  of  the  estimator  (6.29)  in  the  presence  of  the 
disturbances  d^  which  belong  to  the  known  set  of  equation  (6.32).  Since 
the  estimate  x^  is  generated  by  the  controller  and  known  to  him  at  each 
time  k,  we  are  faced  with  a  target  tube  reachability  problem  with  perfect 
state  information  of  the  form  that  was  examined  in  Chapter  3. 

The  solution  of  this  estimate  reachability  problem  can  be  given  using 

the  results  of  Chapter  3.  Define  analogously  to  equations  (3.5),  through 

/V  a  $ 

(3. 8)  the  effective  target  sets  T^j*  and  the  modified  target  sets  X^  by  the 

equations 


A  jj<  A 

XN  =  XN 

(6. 

33) 

A 

Tk+1 

n 

<  *■> 
x> 

x> 

+ 

Lk+lDk)C^k+l*  » 

k  =  0,  1,  . .  ..N-l 

(6. 

34) 

A# 

Xk  = 

{Jkl(Ak*k 

+  Bkuk)e^k+r  for 

some  u^eU^}  D  X^ 

(6. 

35) 

k  = 

1,2,  . . .,  N 

a*  = 

o 

<8o  l(Ao*o 

,  A  . 

+  B  u  eT.,  for  some  u  eU  ; 
o  o'  1  o  o 

(6 

.36) 

Then  the  target  tube  {Xj,  X2,  . .  . ,  X^}  is  reachable  by  the  state  of  the 
estimator  (6.29)  if  and  only  if 


A  A  sjt 

x  =  OeX 
o  o 
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Si  nee  the  reachability  of  the  target  tube  {&j,  $2,  ....  &N)  is  a  sufficient 
condition  for  the  reachability  of  the  target  tube  {Xj,  X2,  •  .  • .  XN)  by  the 
state  x^  of  the  system  (6.  1)  in  the  presence  of  imperfect  information  we 
have  the  following  proposition. 


Proposition  6.2:  A  sufficient  condition  for  reachability  of  the  target  tube 
{Xj,  X2, . . .  i  Xpq}  is  that 


A  * 
OeX 

o 


where  the  set  is  given  recursively  by  equations  (6,  33)  through  (6.  36). 

The  control  law  that  achieves  reachability  of  the  target  tube  {Xj, 

X2, . . . ,  Xj^}  is  the  one  that  achieves  reachability  of  the  target  tube  {Xj, 
A  A  « 

X2, ....  XN)  by  the  estimate  of  the  estimator  (6.29),  and  it  is  of  the 
form 

uk  =  ^k^Xk^  '  k=0,  1 , . . . ,  N- 1 


A  possible  method  for  obtaining  such  a  control  law  which  in  addition  is 
linear,  is  to  make  use  of  the  ellipsoidal  algorithm  of  Sections  3,  4  in 
Chapter  3  assuming  that  the  sets  and  are,  or  can  be  approximated 
by,  ellipsoids.  In  this  case  we  can  use  the  following  ellipsoid  which 
bounds  the  disturbance  set  of  equation  (6.32) 

Dk  =  *dkl  dy  (1"pk+l)  Sc+l^lc+l |  kCk+l  +  pk+lRk+l^  ldk  -  ^ 

The  inclusion  D^C  can  be  easily  verified  from  equation  (6. 19),  (6.20), 
(6.31),  (6.32).  The  infinite  time  version  of  the  ellipsoidal  algorithm  can 
also  be  used  for  constant  systems  with  constant  constraint  sets  when  the 
final  time  N  approaches  infinity.  In  this  case  the  infinite  time  bounding 
ellipsoid  estimator  will  be  used  to  generate  the  estimate  x^. 
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It  should  be  noted  that  in  the  derivation  of  the  sufficient  condition 

of  Proposition  6.2  we  have  made  several  weakening  assumptions.  We 

assumed  that  the  controller  uses  the  bounding  ellipsoid  estimator  to  pro- 

£ 

duce  the  estimate  set  X^|  ^  whereas  the  controller  could  conceivably  cal¬ 
culate  the  exact  set  of  possible  states  X^^*  Furthermore  we  have  not 
taken  advantage  of  the  possibility  to  obtain  a  smaller  estimate  set  by  using 
the  term  6  (k)  in  equation  (6.18).  A  stronger  sufficient  condition  for 
reachability  can  be  obtained  by  making  use  of  this  term.  However  the 
additional  complexity  which  would  be  introduced  would  make  the  resulting 
control  scheme  impractical.  On  the  other  hand  our  approach  to  reduce  the 
state  reachability  problem  with  imperfect  information  to  an  estimate  reach¬ 
ability  problem  with  perfect  information  results  in  a  control  scheme  the 
implementation  of  which  presents  no  more  difficulty  than  the  one  considered 
in  Chapter  3. 

4.  Discussion  and  Sources 

In  this  chapter  two  reachability  problems  with  imperfect  state  infor¬ 
mation  were  examined,  both  involving  a  linear  system.  The  first  problem 
is  a  target  set  reachability  problem  where  the  constraint  set  for  the  uncertain 
quantities  is  specified  by  an  energy  constraint.  For  this  problem  we  char¬ 
acterized  completely  the  optimal  control  law.  This  control  law  is  in  general 
quite  difficult  to  implement,  although  suboptimal  schemes  can  be  derived 
which  can  be  implemented  more  practically.  However  our  primary  objective 
has  been  to  demonstrate  that,  for  this  problem,  the  state  reachability  problem 
with  imperfect  information  is  equivalent  to  an  estimate  reachability  problem 
with  perfect  information,  where  the  estimate  is  generated  by  an  estimator 
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derived  earlier  in  Chapter  4.  This  equivalence  should  be  expected  to  hold 
in  some  form  for  more  general  reachability  problems  not  involving  an 
energy  constraint  for  the  uncertain  quantities.  However  in  such  cases  the 
necessary  estimator,  which  will  produce  some  estimate  set,  will  in  gen¬ 
eral  require  a  very  complicated  implementation. 

The  second  problem  considered  is  a  target  tube  reachability  problem 
with  instantaneous  ellipsoidal  constraints  for  the  uncertain  quantities.  This 
problem  was  first  examined  in  (Bl).  Our  objective  in  this  problem  was  to 
obtain  suboptimal  control  schemes  that  can  be  practically  implemented. 

We  achieved  this  by  reducing  the  state  reachability  problem  with  imperfect 
state  information  to  a  state  estimate  reachability  problem  of  perfect  infor¬ 
mation.  The  state  estimate  is  produced  by  the  (suboptimal)  bounding  ellip¬ 
soid  filtering  algorithm  developed  earlier  in  Chapter  4.  The  resulting  con¬ 
trol  law,  though  obtained  through  substantial  approximations,  involves  no 
more  difficulty  in  its  implementation  than  the  corresponding  perfect  infor¬ 
mation  control  law  considered  in  Chapter  3.  This  control  law  can  be  easily 
modified  for  the  case  where  the  constraint  sets  for  the  uncertain  quantities 
are  defined  in  a  somewhat  different  form  than  those  considered  in  Section  3. 
Such  for  example  is  the  case  where  the  ellipsoids  specified  by  the  constraints 
(6. 17)  are  not  centered  at  the  origin, .  or  the  case  where  there  are  no  mea¬ 
surement  disturbances.  In  these  cases  appropriate  modifications  must  be 
made  in  the  bounding  ellipsoid  estimator.  Another  case  sometimes  appearing 
in  practice  is  when  the  measurements  are  received  by  the  controller  with 
some  delay.  Thus  in  time  i  the  controller  may  know  the  measurements  only 
up  to  time  k,  k  <  i.  Under  these  circumstances  the  controller  must  use  a 
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predictor  to  generate  the  estimate 


instead  of  the  filter  of  Proposition 


4.  3,  and  the  corresponding  estimate  reachability  problem  must  be  ap¬ 


propriately  redefined. 


CONCLUSIONS 


In  this  thesis  the  subject  of  the  feedback  control  of  uncertain  sys¬ 
tems  has  been  examined  for  the  case  where  the  uncertainty  has  nonsto¬ 
chastic  description.  In  addition  to  theoretical  investigations  an  effort 
has  been  made  to  provide  design  algorithms  for  the  feedback  control  of 
uncertain  systems  which  have  potential  for  practical  implementation.  The 
set-membership  description  of  the  uncertain  quantities  appears  to  be  at¬ 
tractive  from  the  point  of  view  of  the  designer  who  is  often  faced  with  a 
situation  where  he  has  an  incomplete  statistical  description  of  the  uncertain 
quantities.  In  such  cases  the  designer  often  subjectively  and  arbitrarily 
assigns  a  probabilistic  description  to  the  uncertain  quantities  with  the  pos¬ 
sible  result  of  a  poor  mathematical  model  for  the  physical  problem  in  hand. 
It  is  the  author's  belief  that  some  of  the  algorithms  in  this  thesis,  partic¬ 
ularly  the  ellipsoidal  algorithms  of  Chapters  3  and  4,  can  provide  a  serious 
alternative  to  such  a  procedure  when  the  set-membership  description  of  the 
uncertainty  is  available.  This  is  particularly  so  since,  where  applicable, 
these  ellipsoidal  algorithms  lead  to  designs  which  have  desirable  features 
from  the  engineering  viewpoint.  The  estimators  of  Chapter  4  have  the  same 
structure  as  linear  minimum  variance  stochastic  estimators,  a  structure 
which  is  desirable  from  the  implementation  point  of  view,  and  the  ellipsoidal 
algorithm  of  Chapter  3  leads  to  a  linear  control  law  which  again  can  be  im¬ 
plemented  with  relative  ease. 

When  considering  the  minimax  approach  towards  a  problem  of  deci¬ 
sion  under  uncertainty  one  should  constantly  be  aware  of  the  fact  that  this 
approach  is  conservative  in  nature.  In  some  problems  where  specified 
performance  tolerances  must  be  met  with  certainty  the  minimax  approach 
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is  the  natural  one.  However  in  many  other  problems  the  minimax  ap¬ 
proach  may  lead  to  unduly  conservative  designs,  and  the  algorithms  pro¬ 
posed  in  this  thesis  should  be  viewed  in  the  light  of  this  consideration.  If 
the  design  obtained  through  the  minimax  approach  is  deemed  too  conser- 
ative,  other  approaches  such  as  a  stochastic  formulation  of  the  problem 
can  be  considered. 

Chapters  2  and  5  of  the  thesis  are  primarily  of  theoretical  nature, 
and  contain  results  which  require,  in  general,  a  substantial  computational 
effort  for  their  use  in  a  practical  situation.  They  are  important,  however, 
for  providing  a  general  framework  for  considering  minimax  problems,  for 
obtaining  existence  results,  for  providing  insight  into  the  structure  of  the 
optimal  controller,  and  for  yielding  results  in  special  cases  such  as  some 
of  those  considered  in  Chapters  3  and  6.  On  the  other  hand,  the  emphasis 
in  Chapters  3  and  4  and  in  part  of  Chapter  6  is  in  the  development  of  algo¬ 
rithms  which  have  potential  for  practical  applications.  These  algorithms 
are  applicable  to  the  case  of  a  linear  discrete-time  system  where  the  con¬ 
straint  seta  for  the  uncertain  quantities  are,  or  can  be  approximated  by, 
ellipsoids.  Although  these  algorithms  have  some  attractive  features,  their 
performance  has  not  as  yet  been  sufficiently  evaluated  either  analytically 
or  by  simulation.  Furthermore  the  question  of  the  quality  of  the  approxi¬ 
mation  involved  in  these  algorithms  remains  as  yet  unresolved.  Thus 
some  research  and  simulations  are  required  to  provide  more  insight  into 
the  merits  and  the  drawbacks  of  these  algorithms. 

Other  areas  where  further  research  is  required  are  the  situations 
where  the  system  is  nonlinear  and/or  continuous-time.  The  feedback  control 
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problem  which  involves  a  nonlinear  system  and  a  set-membership  des¬ 
cription  of  the  uncertainty  requires,  in  general,  excessive  computational 
effort  for  its  solution  as  discussed  in  Chapter  5.  In  this  area  optimal  or 
nearly-optimal  algorithms  that  are  computationally  feasible  can  be  ex¬ 
pected  only  for  problems  with  special  structure.  The  same  appears  to 
be  true  for  the  state  estimation  problem  involving  a  nonlinear  system. 

The  state  estimation  problem  involving  a  continuous -time  linear  system 
and  either  an  energy  constraint  or  instantaneous  ellipsoidal  constraints  on 
the  uncertain  quantities  presents  no  more  difficulty  than  its  discrete-time 
counterpart  and  has  been  considered  in  (B2).  The  feedback  control  prob¬ 
lem  involving  a  continuous -time  system  appears  to  present  considerably 
greater  technical  difficulties  than  its  discrete-time  counterpart.  This 
problem  is  essentially  a  differential  game  for  which  a  saddle  point  in 
pure  strategies  is  not  necessarily  assumed  to  exist,  and  is  worthy  of 
careful  consideration. 

Finally  is  should  be  mentioned  that  while  in  general  the  complete 
characterisation  of  the  optimal  controller  ina  minimax  control  problem  is 
a  very  difficult  task,  the  same  is  true  of  stochastic  optimal  control  prob¬ 
lems  with  the  exception  of  the  case  of  the  separation  theorem  for  linear 
systems  and  quadratic  criteria.  Unfortunately  no  result  comparable  in 
elegance  to  the  stochastic  separation  theorem  appears  to  exist  in  connec¬ 
tion  with  any  particular  minimax  control  problem. 


APPENDIX  I 


ON  THE  THEORY  OF  CONVEX  FUNCTIONS 

In  this  appendix  some  definitions  and  results  are  summarized 
concerning  convex  functions  defined  on  a  finite  dimensional  Euclidean 
space.  Only  the  results  that  are  necessary  for  the  developments  in 
Chapter  2  are  presented  here.  A  complete  exposition  of  the  theory  can 
be  found  in  (Rl). 

The  range  of  the  function  that  we  will  be  concerned  with  is  the 
extended  real  line  R^  -  l-oo,  oo].  The  conventions  adopted  concerning 
arithmetic  operations  on  R0  which  involve  -ao  or  oo  are  as  follows: 
Concerning  addition  we  have: 
a  +  cd  =  oo  +  a  =  ao,  for  ae  (-ao,  oo] 
a  -  ao  -  -ao  +  a  -  -oo,  for  ae  [-00,  00) 

Concerning  multiplication  we  have: 
a  ao  -  ao  a  =  ao,  a(-oo)  =  (-oo)a  =  -00,  for  ae  (0,  ao] 
a  ao  =  ao  a  =  -oo,  a(-ao)  =  (-ao)a  =  ao,  forae[-ao,0) 

0 ao  =  00  0  =  0  -  0(-oo)  =  (-oo)O 

The  sums  ao  -  ao,  -ao  -too  are  undefined  and  are  avoided.  Under  these 
rules  addition  and  multiplication  are  commutative  and  associative,  and  the 
distributive  law 

a(aj  +  a2)  =  aa}  +  aa2 

holds  provided  the  sum  (a^  t  a2)  is  neither  of  the  forbidden  sums  ao  -  ao  and 
-ao  +  ao. 

The  cancellation  laws  hold  as  follows 
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a  +  aj  =  a  +  a2  aj  =  a 2.  for  a  4  -oo,  oo 

aa,  =  aa^  a,  =  a^,  for  a.  4  0,  -oo,  ao 

Order  on  the  extended  real  line  is  defined  in  the  natural  way. 
Concerning  inequalities  we  have  the  cancellation  laws: 

a  +  aj  <  a  +  a2  aj  <  a2,  for  a  V  -oo,  oo 

a*j  1  **2  al  —  a2’  *or  af  (O.oo) 
aaj  <  it2  aj  >  a^,  for  ac  (-ao,  0) 

One  of  the  advantages  of  the  extended  real  line  is  that  it  is  closed 
under  taking  the  supremum  or  the  infimum  of  any  of  its  subsets  with  the 
additional  convention  that  for  the  empty  set 


inf  $  =  ao ,  sup  ^  =  -oo 
The  familiar  minimax  inequality 

sup  inf^(x,  y)  <  inf  sup^  (x,  y) 
ye  Y  xe  X  xeX  ye  Y 

holds  for  any  function^  :  Rn  x  Rm  —  (  -oo,  oo]  and  any  sets  X  CRn,  YCR™. 

In  calculations  which  involve  the  supremum  or  infimum  operation 
care  sometimes  must  be  exercised  so  that  the  forbidden  sums  oo  -  oo, 

-oo  +  ao  do  not  appear.  For  example  if  fj  :Rn  —  (-oo,  oo],  f^  :Rm  — 

(-ao,  ao)  are  functions  and  X,  Y  are  subsets  of  Rn  and  Rm  respectively 
we  have 


inf  l  fj(x)  +  f2(y)]  =  inf  inf  (  fj (x)  +  f2(y)] 
x,  y  xfX  ytY 

xc  X,  y«  Y 

=  inf  ( f.(x)  +  inf  f?(y)] 
xcX  1  yeY 
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only  if  either  -oo  <  inf  f-(y)  or  f,(x)  <  co,  V  xeX. 

y€Y  2  1 

Such  calculations  are  common  in  dynamic  programming  algorithms 
and  will  be  used  frequently  in  the  text  of  the  thesis. 

We  now  introduce  some  of  the  notions  related  to  convex  functions. 
Let  f  :Rn  —  [  -oo,  oo)  be  a  function. 

Definition  A.  I :  The  epigraph  of  f  is  the  subset  of  Rn+* 
epif  =  {(x,  p)|xeRn,  peR,  p  >  f(x)} 

Definition  A. 2;  The  function  f  is  called  convex  if  the  set  epif  C  Rn+*  is 
convex.  If  -oo  <  f(x),  V  x<  Rn  this  is  equivalent  to 

fid -\)x  +  \y]  <  (1  -X)f(x)+\f(y),  V  \€  (0,  1),  V  x ,ye  Rn 

Definition  A.  3:  The  convex  hull  of  a  function  f,  denoted  by  convf,  is  the 
convex  function  which  has  as  epigraph  the  set  conv  (epif)  (convex  hull  of 
epif). 

Definition  A. 4:  A  convex  function  f  is  said  to  be  proper  if  -ao  <  f(x), 

V xf  R  4  and  f(x)  <  ao  for  at  least  one  xc  Rn.  It  is  said  to  be  closed  if  epif 
is  a  closed  set. 

Definition  A.  5:  The  closure  of  a  proper  convex  function  f,  denoted  by  elf, 
is  the  closed  proper  convex  function  which  has  as  epigraph  the  set  cl(epif) 
(closure  of  the  set  epif). 

Concerning  closed  proper  convex  functions  we  have  the  following 
proposition: 


Proposition  A.  1 :  Let  f  be  a  convex  proper  function.  The  following 
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conditions  are  equivalent 

(a)  f  is  closed. 

(b)  The  level  sets  {x|f(x}  <  a}  are  closed,  y  ae  R. 

(c)  f  is  lower  semicontinuous. 

Proof:  See  Theorem  7.1  in  Reference  (Rl). 

Definition  A.  6:  The  effective  domain  of  a  convex  function  f  is  the  convex 
set 

domf  =  (x|f(x)  <  00} 

A  subset  L  of  Rn  is  called  affine  (linear  manifold)  if  (1  -  \  )  x  +  Xye  L, 
^x,  ye  L,  V  \(  R.  Given  now  a  convex  set  C  in  Rn  the  affine  hull  of  C  is  the 
smallest  affine  set  that  contains  C.  With  these  definitions  we  have: 

Definition  A.  7 :  The  relative  interior  of  the  effective  domain  of  a  convex 
function  f,  denoted  ri  (domf),  is  the  interior  of  the  set  domf  relative  to  its 
affine  hull. 

Definition  A.  8:  A  convex  function  f  is  said  to  be  positively  homogeneous  if 

f(Xx)  =  Xf(x),  V  xe  Rn ,  yxc(0,oo) 

An  example  of  a  positively  homogeneous  convex  function  is  the 
support  function  of  a  convex  set  C 

r(x|C)  =  sup  <x,  x*> 
x*<  C 

Concerning  continuity  of  convex  functions  we  have: 

Proposition  A. 2:  Let  f  be  a  proper  convex  function  on  Rn.  Then  the  re¬ 
striction  of  f  to  any  subset  C  of  domf  which  is  open  relative  to  the  affine 
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hull  of  domf  is  continuous.  In  particular  the  restriction  of  f  to  ri  (domi) 
is  continuous.  This  implies  that  a  convex  function  which  is  finite  on  all 
of  Rn  is  continuous. 

Proof:  See  Theorem  10.  1  of  Reference  (Rl). 

Some  important  operations  involving  convex  functions  will  now 
be  introduced: 

Proposition  A.  3:  If  fj,  f^  are  proper  convex  functions  in  Rn  the  function 
fj  +f^  is  convex.  It  is  proper  if  domfj  0  dom  f^  4  $ . 

Proof:  See  Theorem  5.2  in  Reference  (Rl). 

Proposition  A.  4:  If  f  j ,  f^  are  proper  convex  functions  in  Rn  the  function 
f  defined  by 

f(x)  =  inf  {fj(x  -  y)  +  f2(y)} 

y 

is  convex.  The  function  f  is  denoted  as  f  =  f  j  □  f^  and  the  operation  □ 
is  called  infimal  convolution. 

Proof:  See  Theorem  5.4  in  Reference  (Rl). 

Proposition  A.  5:  Let  f.,  if  I,  be  convex  functions,  where  I  is  an  arbitrary 
inds-'x  set.  Then  the  function  f  defined  by 

f(x)  -  sup  f  (x) 
if  I  1 

is  convex. 

Proof:  See  Theorem  5.5  in  Reference  (Rl). 

Proposition  A ■  fe:  Let  A  be  a  linear  transformation  from  Rn  to  Rm.  Then 
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for  each  convex  function  g  on  Rm,  the  function  gA  defined  by 
(gA)(x)  =  g(Ax) 

is  convex  in  Rn.  For  each  convex  function  h  on  Rn,  the  function  Ah 
defined  by 

(Ah)(y)  =  inf  h(x) 

Ax=y 

is  convex  on  Rm  (Notice  that  in  accordance  with  the  convention  inf  ^>=00 
we  have  (Ah)(y)  =  00  for  all  y  which  are  not  in  the  range  of  A).  The  func¬ 
tion  Ah  is  called  the  image  of  h  under  A  and  the  function  gA  is  called  the 
inverse  image  of  g  under  A. 

Proof:  See  Theorem  5.7  in  Reference  (Rl). 

We  now  introduce  the  notion  of  the  recession  function  of  a  convex 
function.  This  notion  is  extremely  helpful  in  proving  closure  and  properness 
of  functions  resulting  from  functional  operations  introduced  earlier  as  well 
as  in  proving  existence  of  solutions  in  convex  optimization  problems. 

Let  C  be  a  nonempty  convex  set  in  Rn.  We  say  that  C  recedes  in 
the  direction  of  the  vector  s  V  0  if  and  only  if  x+Vte  C  for  every  X  >  0  and 
x<r  C.  The  set  of  all  vectors  zc  Rn  that  satisfy  this  condition  together  with 
z  =  0  is  called  the  recession  cone  of  C,  denoted  by  0+C. 

Definition  A.  9:  Let  f  be  a  proper  convex  function.  The  recession  function 
f0+  of  f  is  the  convex  function  which  has  as  epigraph  the  set  0+(epi  f). 

Proposition  A .  7 :  The  recession  function  f  0+  of  a  proper  convex  function  f 


is  a  positively  homogeneous  proper  convex  function  given  by 
(fO+)(z)  =  sup  {f(x  +  z)  -  f(x) | xe  dom f} 
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If  f  is  closed  then  f0+  is  closed  also. 

Proof:  See  Theorem  8.5  in  Reference  (Rl). 

Definition  A.  1 0:  A  direction  defined  by  a  vector  z  4  0  is  called  a  direction 
of  recession  of  the  proper  convex  function  f  if  (f0+)(z)  <  0.  It  is  called  a 
direction  in  which  f  is  constant  if  (fO+)(z)  =  (f0+)(-z)  =  0. 

Thus  a  proper  convex  function  f  every  direction  of  recession  of  which  is  a  direc¬ 
tion  in  which  it  is  constant  is  characterized  by  the  fact  that  (f0+)(  z)  >  0, 
and  (f0+)(x  )  =  0  implies  (f0+)(-  z)  =  0. 

Some  criteria  for  properness  and  closure  of  functions  resulting  from 
functional  operations  of  convex  functions  will  now  be  given. 

Proposition  A.  8:  If  fj,  f 2  are  closed  proper  convex  functions  and  fj  +  f2  is 
not  identically  ao  then  fj  +  f2  is  a  closed  proper  convex  function  and 

(f1+f2)0+  =  fj0+  +  f20+ 

Proof:  See  Theorem  9.3  in  Reference  (Rl). 

Proposition  A.  9:  Let  fj,  f2  be  closed  proper  convex  functions  in  Rn  such 
that  there  exists  no  z€  Rn  such  that 

(f10+)(z)  +  (f20+)(-z)  >  0 

(fj0+)(-z)  +  (f20+)(»)  <  0 

Then  f ^  is  a  closed  proper  convex  function  and  the  infimum  in  the 
equation 

(fj  □  f2)(x)  =  inf  {fj(x-y)  +  f2(y)} 

y 

is  attained  by  some  y  for  each  x.  Moreover 
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(fld£2)0+  =  f,0+df20+ 

Proof:  See  Corollary  9.2.1  in  Reference  (Rl). 

Proposition  A.  10:  Let  L,  id,  be  closed  proper  convex  functions  where 
I  is  an  arbitrary  index  set.  Then  the  function  f  defined  by 

f(x)  =  sup  f.(x) 
id  1 

either  is  the  constant  oo  function  or  it  is  a  closed  proper  convex  function 
and  f0+  is  given  by 

(f0+)(z)  =  sup(f.0+)(z) 
id  1 

Proof:  See  Theorem  9.4  in  Reference  (Rl). 

Proposition  A.  1 1 :  Let  h  be  a  closed  proper  convex  function  on  Rn,  and 
let  A  be  a  linear  transformation  from  Rn  to  Rm.  Assume  that  there  exists 
no  z€  Rn  such  that  Az  =  0,  (h0+)(z)  <  0  and  (h0+)(-z)  >  0.  Then  the  func¬ 
tion  Ah,  where 

(Ah)(y)  =  inf  h(x) 

Ax=y 

is  a  closed  proper  convex  function  and  (Ah)0+  =  A(h0+).  Moreover  for 
every  y  such  that  (Ah)(y)  <  oo  the  infimum  in  the  definition  of  Ah  is  attained 
for  some  x. 

Proof:  See  Theorem  9.2  in  Reference  (Rl). 

Proposition  A.  12:  Let  g  be  closed  proper  convex  function  on  Rm,  and  let  A 
be  a  linear  transformation  from  Rn  to  Rm.  Assume  that  the  function  gA 
defined  by 
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(gA)(x)  =  g(Ax) 


is  not  identically  oo.  Then  gA  is  a  closed  proper  convex  function  and 
(gA)0+  =  (gO+)A. 

Proof:  See  Theorem  9.5  in  Reference  (Rl). 


As  an  application  of  the  above  propositions  consider  the  function 


in  Rn  defined  by 


Hk(xk)  =  inf  {Ek+l(Vk  +  Bkuk,  +  8k(uk)} 

uk 


(A.  1 ) 


where  :  Rn  (-00,00],  g^  :  Rm  —  (-00,  00]  are  closed  proper  convex 

functions  and  A^  :  Rn  -*  Rn,  B^:Rm  -*  Rn  are  given  linear  transformations. 
The  function  above  is  of  great  interest  in  Chapter  2.  Assume  that  is 
not  identically  00  and  consider  the  following  assumptions: 


Assumption  R:  Every  direction  of  recession  of  each  of  the  functions  Ek+J 
and  g^  is  a  direction  in  which  this  function  is  constant. 

Assumption  C:  The  recession  function  of  g^  is  of  the  form 

(gk0+)(*)  =  oo  for  z  4  0,  (gk0+)(0)  =  0 

The  following  proposition  holds: 

Proposition  A.  13:  Under  either  Assumption  R  or  Assumption  C  the  function 
of  equation  (A.  1 )  is  given  by 

Hk  =  iEk+iO<-WAk  <A-2> 

and  it  is  a  closed  proper  convex  function.  Furthermore  for  each  x^e  Rn  the 
infimum  in  equation  (A.  1)  is  attained  by  some  u^e  Rm,  and  in  the  case  of 
Assumption  R  every  direction  of  recession  of  the  function  is  a  direction 
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in  which  H,  is  constant, 
k 

Proof:  Under  either  Assumption  R  or  Assumption  C  the  conditions  of 
Proposition  A.  11  are  satisfied  so  that  the  function  (-B^lg^  given  by: 

K-aJgJM  =  inf  gk(u.  > 

y-w 

is  a  closed  proper  convex  function  and  ( (-B^lg^l  0+  =  (-B^lg^O*. 

We  have  now 

Hk<xk>  1  inf  {Ekti'Vk +  Bkuk> +  *k<uk)} 

K 

*  inf  inf  {Ek+i'Akxk- yi^kM 
y  "k 

'R-Bkuk 

*  inf  {Ekt,(Akxk-y)+  inf  gk(uk)) 

y  uk 

y-  -Bk"k 

’  inf  {Ek+1(Akxk-y)+  <-Bk)g|[  (y)} 

=  tEk+l  °<-Bk>*kJ<Akxk> 

*  ['Ek+lD'-Bk'«klAk]<xk> 


where  by  our  assumptions  none  of  the  forbidden  sums  oo  -  oo  or  -oo  +oo 
appears  in  the  above  algebra.  Thus  equation  (A.  2)  is  proved. 

The  relation  [(-B^g^lO*  =  (-Bj?)g^0+  implies  in  the  case  of 
Assumption  R  that  every  direction  of  recession  of  the  function  (-B^lg^  i® 
a  direction  in  which  this  function  is  constant  as  can  be  easily  seen  by 
applying  the  appropriate  definitions  and  Proposition  A.  11.  In  the  case  of 
Assumption  C  we  have 
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l  (-Bk)gk]  0+  (z)  =  oo  for  z  4  0,  [  (-Bk)gk]  0+  (0)  -  0. 

In  both  cases  the  conditions  of  Proposition  A.  9  are  satisfied  so  that 
the  function  Ek+jD(-Bk)gk  is  a  closed  proper  convex  function  and  by  Propo¬ 
sition  A.  12  the  function  Hk  of  equation  (A.  2)  is  also  a  closed  proper  convex 
function. 

To  see  that  the  infimum  in  equation  (A.  1)  is  attained  observe  that 
if  xk  is  such  that  Hk(xk)  =  ao  then  the  infimum  is  attained  for  every  uke  Rn. 
If  xk  is  such  that  Hk(xk)  <  ao  then  attainement  of  the  infimum  follows  by 
making  use  of  the  conclusions  of  Propositions  A.  9  and  A.  11  in  the  equation 


Hk(xk)  -  inf  {Ek+1(Akxk-y)+  inf  gk(uk)} 

y  uk 

y=_Bkuk 

In  the  case  of  Assumption  R  the  conclusion  that  every  direction  of 
recession  of  the  function  Hk  is  a  direction  in  which  it  is  constant  follows 
from  the  equation 

Hk0+  '  lEk+l0+O<-Bk>«k0+lAk 


which  holds  by  Propositions  A.  9.  A.  1 1  and  A.  12,  Q.  E.  D. 

The  important  notion  of  the  conjugate  function  of  a  convex  function 
is  now  introduced. 

Definition  A.  1 1 :  Let  f  be  a  convex  function  in  Rn.  The  conjugate  function 
f  is  defined  as 

fV>  -  sup  {<  x,x*  >  -f(x)}  =  -  inf  (f(x)  -  <  x,  x*  >} 

X  X 

and  is  a  closed  convex  function  in  Rn,  proper  if  and  only  if  f  is  proper. 
Moreover  (elf)  -  f  and  (f  )  =  elf. 
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An  important  example  of  a  pair  of  conjugate  functions  is  the  indicator 
function  and  the  support  function  of  a  closed  convex  set  C.  We  have 

6(x|C)  =  0  if  xeC,  6(x|C)  =  oo  if  x^X 

o*{x*  |C)  =  <r(x*|C)  =  sup  <  x*,  x  > 

xeC 

Concerning  conjugate  functions  of  convex  functions  resulting  from  the 
operations  introduced  earlier  we  have  a  duality  between  addition  and  infimal 
convolution,  and  between  the  image  and  the  inverse  image  of  a  convex  func¬ 
tion  under  a  linear  transformation,  in  accordance  with  the  following  propo¬ 
sitions: 

Proposition  A.  14:  Let  fj,  be  proper  convex  functions  in  Rn.  Then 
(f,0f2>*  .  f*  +  f2* 

(cl  fj  +  clf2>*  =  cl(f  *0f*> 

and  if  ri(dom f j ) Ori(dom f^)  4  <p  the  closure  operation  can  be  ommited  from 
the  second  equation. 

Proof:  See  Theorem  16.4  in  Reference  (Rl). 

Proposition  A.  5:  Let  A  be  a  linear  transformation  from  Rn  to  Rm.  For 
any  convex  function  h  on  Rn  we  have 

(Ah)*  =  h*A' 

For  any  convex  function  g  on  Rm  we  have 
( (clg)A)  *  =  cl  (A'g*) 

and  if  there  exists  an  x  such  that  Axe  ri  (domg),  the  closure  operation  can 
be  ommited  from  the  second  equation. 
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Proof:  See  Theorem  16.3  in  Reference  (Rl). 

The  notion  of  the  directional  derivative  and  the  related  notion  of 
the  subdifferential  is  fundamental  in  the  problem  of  finding  the  minimum 
of  a  convex  function. 

Let  f  be  any  function  from  Rn  to  [  -oo,  oo]  and  let  x  be  a  point  where 
f  is  finite.  The  one-sided  directional  derivative  of  f  at  x  with  respect  to  a 
vector  ye  Rn  is  defined  as  the  limit 

r (»;y>  *  Um  Uzthjll '  (A.  3) 

X— 0 

if  it  exists  (oo  and  -ao  being  allowed  as  limits). 

Proposition  A.  16:  Let  f  be  a  convex  function,  and  let  x  be  a  point  where  f 
is  finite.  Then  for  every  y  the  limit  in  equation  (A.  3)  exists  and  the  func¬ 
tion  T(x;y)  is  a  positively  homogeneous  convex  function  of  y. 

Proof:  See  Theorem  23.  1  in  Reference  (Rl). 

Definition  A.  12:  A  vector  x* €  Rn  is  said  to  be  a  subgradient  of  a  convex 
function  f  in  Rn  at  a  point  x  if 

f(z)  >  f(x)  +  <x  ,  z  -  x  >,  V  *C  Rn  (A.  4) 

The  set  of  all  subgradients  of  f  at  x  is  called  the  subdifferential  of  f  at  x, 
it  is  denoted  by  ^f(x),and  it  is  a  closed  convex  set.  If  df(x)  4  $  then  f  is 
said  to  be  subdifferentiable  at  x. 

Proposition  A.  17 :  Let  f  be  a  proper  convex  function.  For  xfdomf^f(x)  is 
empty.  For  xcri(domf),  ^f(x)  is  nonempty  and  f'(x;y)  is  the  support  func¬ 
tion  of  M(x).  Finally  ^f(x)  is  a  compact  set  if  and  only  if  xcint(domf). 
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Proof:  See  Theorem  23.4  in  Reference  (Rl). 

It  should  be  noted  that  if  f  is  differentiable  at  a  point  x  then  ^f(x) 
consists  of  a  single  point  the  gradient  Vf(x).  Another  important  case  to 
note  is  when  f  is  the  indicator  function  of  a  closed  convex  set  C.  Then 
for  xc  C,  df(x)  =  S6(x|  C)  =  (x*|  0  >  <  x*,  z  -  x  >,  ^  zeC} ,  i.e.,  ^6(x|C) 

is  the  set  of  all  vectors  normal  to  C  at  x. 

Duality  is  prevalent  in  the  theory  of  subgradients  due  to  the  following 

fact- 

Proposition  A.  18:  For  any  proper  convex  function  f  and  any  x  the  following 
conditions  are  equivalent: 

(a)  x%df(x) 

* 

(b)  <  z,x  >  -  f(z)  achieves  its  supremum  in  z  at  z  =  x. 

(c)  f(x)  +  fV)  =  <x.x*> 

Proof:  See  Theorem  23.  5  in  Reference  (Rl). 

We  also  have: 

Proposition  A.  19:  Let  fj,  f^  be  proper  convex  functions  on  Rn  and  let 
f  =  fj  +  f Then 

df(x)  ZD  ^fj(x)  +  ^f2(x)  ,  V xff Rn 

If  ri(domf|)  fl  ri(domf2)  4  then  actually 

df(x)  =  ^fj(x)  +  ^f2(x)  ,  V  xf Rn 

Proof:  See  Theorem  23.8  in  Reference  (Rl). 

Proposition  A.  20:  Let  f(x)  =  h(Ax),  where  n  is  a  proper  convex  function 
on  Rm  and  A  a  linear  transformation  from  Rn  to  Rm.  Then 
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^f(x)  ^  A'Sh(Ax),  *xeRn 

If  the  range  of  A  contains  a  point  of  ri  (dom  h)  then 

df(x)  =  A'dh(Ax),  Vx€  Rn 

Proof:  See  Theorem  23.9  in  Reference  (Rl). 

If  {xn)  is  a  sequence  converging  to  a  point  xcRn  it  is  not  generally 
true  that  for  any  yeRn  the  sequence  {f'(xn;y)}  converges  to  f'(x;y).  The 
following  proposition  however  is  useful  in  some  cases. 

Proposition  A. 21 :  Let  f  be  a  convex  function  on  Rn  and  let  C  be  an  open 
convex  set  on  which  f  is  finite.  Let  {f^}  be  a  sequence  of  convex  functions 
finite  on  C  and  converging  pointwise  to  f  on  C.  Let  xe  C  and  let  {xn}  be  a 
sequence  of  points  in  C  converging  to  x.  Then  for  any  yc  Rn  and  any  sequence 
{yn}  converging  to  y  we  have 

lirr  sup  f'n(xn;  yR)  <  f'(x;y) 
n  — oo 

Proof:  See  Theorem  24.5  in  Reference  (Rl). 

As  an  application  of  the  above  we  prove  the  following  proposition 
which  will  be  useful  in  Chapter  2.  Th' t  proposition  is  a  generalization  of 
results  in  References  (Dl)  and  (Dal). 

Proposition  A.  22 :  Let  :  Rn  x  Rm  —  (-ao,  oo)  be  a  function  and  let  Y  be  a 
compact  subset  of  Rm.  Assume  further  that  for  every  vector  ycY  the  func¬ 
tion  ^  (•  ,  y) :  Rn  —  (-oo,  oo]  is  a  closed  proper  convex  function.  Consider 
the  function  f  defined  as 

f(x)  -  sup$(x,y) 
y«Y 
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Then  if  f  is  finite  somewhere  it  is  a  closed  proper  convex  function. 
Furthermore  if  int(dom  f)  4  $  and  (f  is  continuous  on  the  set  int(dom  f)  x  / 
then  for  every  xe  int(domf)  we  have 

df(x)  =  conv  {djf(x,  y)|ye  Y(x)} 

where  Y(x)  is  the  set 

Y(x)  =  (9«Y|£{x,9)  =  max ^  (x,  y)} 

Proof:  The  fact  that  f  is  a  closed  proper  convex  function  follows  immedi¬ 
ately  from  Proposition  A.  10.  Let  xcint(domf)  and  let  C  be  an  open  convex 
neighborhood  of  x  such  that  f  is  finite  on  C.  Then^J  is  finite  and  continuous 
on  CxY  since  CCint(domf).  Now  for  any  *eRn  let  {x^}  =  {x  +  X^z}  be  a 
sequence  of  vectors  in  C  with  X  >  0,  {X  }  — >  0.  Let  also  {y  }  be  a  se- 
quence  of  vectors  such  that  f(x^=  y  (xn,  yn),  i.e.,  ynfY(xn)  C  Y.  Such  a 
sequence  exists  by  the  compactness  of  Y  and  the  continuity  of  the  function 
y  (x^* )  on  Y.  Furthermore  by  compactness  of  Y  the  sequence  {y^}  has  a 
subsequence  which  converges  to  a  vector  ycY.  Thus  without  loss  of  gener¬ 
ality  we  can  assume  that  the  sequence  i*n)  is  selected  so  that  the  corre¬ 
sponding  sequence  (yn)  converges  to  the  vector  ye  Y.  We  now  prove  that 
in  fact  ye  Y(x),  i.e.,  thatfjf  (x,  y)  =  max^(x,y).  We  have  by  the  continuity 
of  f  on  C  for  any  C  >  0 


f(x)  -  *  <  f(xn) 


for  n  >  Nj 


and  by  the  continuity  of  on  CxY 

f(xn)  (xn»yn)  5  y<x»y)  +  c 


for  n  > 


and  therefore 
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f(x)  =  m&x(f(x,y)  y)  +  2e 

Y*  X 

and  since  the  above  inequality  holds  for  any  e  >  0  we  conclude  maxCf(x,  y)  = 

ye  Y  *> 

Cf(x,  y)  and  ye  Y(x). 

Now  we  have: 

f(x+Xnz)  -  f(x)  y(»»y.yn)  -ytx.yi 


0(X_) 


with  lim 


<><*  ) 


=  0.  Taking  limits  in  the  above  inequality  we  obtain 


n— oo  n 


f'(x;£)  <  lim  sup(^(x,  yn;z) 
n-*oo 


(A.  5) 


Now  the  sequence  of  functions  of  x  )}  converges  pointwise  to 

the  function^- ,  y)  on  the  open  set  C  by  the  continuity  of  on  CxY,  and  by 


applying  Proposition  A.  21 


lim  supjf  (x,  yn;s)  (x,y;s) 


n— oo 


Using  the  above  in  relation  (A.  5)  we  obtain 

f'(x:  )  5  y'ix.y-.z) 

On  the  other  hand  we  have  for  every  vector  ye  Y(x) 

f{x  +  Xnz)  -  <(x)  y(x  +  \«.y„)  -y<x.y) 

£  =  K 

n  n 

^(*  +  xn*.y)  y> 


(A.  6) 


Taking  limits  in  the  above  inequality  we  obtain 


f'(x;z)  >  y*(x,  y;z),  Vye  Y(x)  (A.  7) 

From  relations  (A.  6)  and  (A.  7)  it  follows  that 

$ 

f(x;z)  =  max  (i(x,  y;z) 
y*Y(x)J 

i 

and  since  by  Proposition  A.  17  ^(x,  y;  * )  is  the  support  function  of  the  convex 
compact  set  ^(x,y)  and  f'(x;* )  is  the  support  function  of  the  convex  compact 
set  df(x)  it  follows: 

df(x)  =  conv  {^yi(x,  y)|y€Y(x)}  Q.E.D. 

Consider  now  a  closed  proper  convex  function  of  f  and  the  problem 
of  finding  its  m.nimum  in  Rn.  The  set  of  points  xe  Rn  such  that 

f(x)  =  inf  f(x) 
x 

will  be  called  the  minimum  set  of  f.  We  have  the  following  proposition: 


Proposition  A.  23 :  The  following  statements  are  valid  for  any  closed  proper 
convex  function  f  and  its  conjugate  f*. 

^  jj, 

(a)  inf  f(x)  =  -f  (0).  Thus  f  is  bounded  below  if  and  only  if  Otdomf  . 

x 

(b)  A  vector  x  belongs  to  the  minimum  set  of  f  if  and  only  if  Oe  b((x). 

$ 

(c)  The  minimum  set  of  f  is  df  (0).  Thus  the  infimum  of  f  is 

attained  if  and  only  if  I*  is  subdifferentiable  at  0.  This  condition 

is  satisfied  in  particular  when  Oe  ri(doinf*);  moreover  one  has 
£ 

Oe  ri(dom  f  )  if  and  only  if  every  direction  of  recession  of  f  is 
a  direction  in  which  f  is  constant. 
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(d)  The  minimum  set  of  f  is  a  nonempty  compact  set  if  and 

$ 

only  if  Oe  int(doxn  f  ).  This  holds  if  and  only  if  f  has  no 
directions  of  recession. 

(e)  The  minimum  set  of  f  consists  of  a  unique  vector  x  if  and 

£  ♦ 
only  if  f  is  differentiable  at  0  and  x  =  Vf  (0). 

Proof:  See  Theorem  27.1  in  Reference  (ill). 

The  above  Proposition  illustrates  the  fundamental  role  of  the  sub¬ 
differential  in  convex  minimization  problems  and  shows  the  importance  of 
the  recession  function  in  such  problems. 


APPENDIX  II 


In  this  Appendix  we  present  the  proofs  of  Propositions  3.  1  and  3.2. 

We  begin  with  the  proof  of  Proposition  3.1.  For  the  purpose  of  clearer 
presentation  a  few  lemmas,  some  of  which  are  well  known,  will  be  given 
first: 

Lemma  A.  1 :  Let  2^,  Z2  be  positive  definite  symmetric  nxn  matrices. 

Then 

(a)  <  Z2  if  and  only  if  2^  >  22* 

(b)  There  exist  positive  scalars  |i,  v  such  that 

V22  <  2,  < 

Proof:  (a)  For  allycR”,  (y'271yK2  =  sup  <x,y>  >  sup  <x,y>  =  (y'Z^y^Q.E.D. 

1  x'Zf<  1  x'Z^x  <  1  C 

(b)  For  any  two  norms  in  Rn,  ||  •  |j|j,  ||*||  2> there  exist  positive  scalars 

p ,  v  such  that 

^1/2||x|I2  <  IMIj  <  P1/2||xtl2  for  all  xeRn 

Taking  ||x||j  =  (x' 2jx)^2,  ||x||^=  (x‘ E^x)1  ^2  the  result 

follows  Q.  E.  D. 

Lemma  A.  2:  Let  F  be  an  nxn  matrix  such  that  for  every  eigenvalue 

2  2 

X  =  a  +  bi  of  F  we  have  a  +b  <1.  Then  there  exists  a  positive  definite 
symmetric  matrix  M  such  that 
F'MF  <  M 

Proof:  This  lemma  is  a  direct  consequence  of  the  fact  that  if  p(F)  = 

max  {Va2  +  b2  |X  =  a  +  bi,  X:eigenvalue  of  F}  then  for  every  e.>  0  there  exists 
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1  /2 

a  Euclidean  norm  ||x||  =  (x'Mx)  such  that 


p(F)  <  |  Jf|  |  =  sup 


Fx  I 


(x'F'MFx) 


1/2 


=  sup 

x  (x'Mx) 


f n—  < 


(see  for  example,  Reference  (II) 

Since  p(F)  <  1  there  exists  a  positive  definite  symmetric  matrix  M  such  that 

sup  <  , 

x  (x'Mx) 


implying  F'MF  <  M  Q.E.D. 


Lemma  A.  3:  Let  {£k}  be  a  sequence  of  positive  definite  symmetric  nxn 
matrices  such  that  <  2k+j  <  M  for  all  k,  where  M  is  a  positive  definite 
symmetric  matrix.  Then  the  sequence  converges  (in  any  norm  in  Rn  ) 

to  a  positive  definite  symmetric  matrix  2^. 


Proof:  This  lemma  is  a  special  case  of  a  result  for  positive  operators  in 
Hilbert  space  and  has  appeared  in  this  form  in  Reference  (Wo2). 


Lemma  A.  4:  Consider  the  sequences  of  matrices  (K^}t  (2k)  generated 
by  the  equations 

Kk  l  =  A'fK"1  -  GQ_1G'  +  ER^B'J^A  +¥  (A.  8) 

K  =V 
n 

2k  l  =  (A-BL)'^1 -GQ_1G')_1(A-BL)  +¥  +  L'RL  (A.  9) 

2  =¥ 
n 

where  A,  B,  G,  L  are  given  matrices  of  dimension  nxn,  nxm,  nxr,  mxn 
respectively  and^.Q  and  R  are  given  positive  definite  symmetric  matrices. 
Assume  that  for  all  k  the  matrix  2k  is  positive  definite  and  symmetric  and 
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that 

GQ^G'  <  S^1  for  all  k 

Then  we  have  for  all  k 

0  <  Kk  <  2k  and  GQ_1G'  <  K"1 

Proof:  We  prove  the  lemma  by  induction.  For  k  =  N  it  holds.  Assume 
0  <  KR  <  ZR.  It  will  be  proved  that  0  <  KR  j  <  ZR  j.  For  convenience 
write 

M  =  (ZR2  -GQ^G'f1  >  (K’1  -GQ'1  .f1 
Then  from  equations  (A.  8)  and  (A.  9) 

2k-l  "  Kk-1  =  (A-BL).M(A-BL)  +¥  +  L'RL 

-  A'fK^1- GQ-1G' +  Bk”1B,)~1  A  V 

>  (A-BL)'M(A-BL)  +  L'RL  -  A'(M_1  +  BR_1B'f1A 
By  using  the  well  known  matrix  identity 

(M"1  +BR'1B')"1  =  M  -  MB(B'MB  +  R)"2  B'M 
in  the  above  inequality  and  by  expanding  we  obtain 

ZR1  -  Kk  J  >  A'MA  +  L'(B'MB+R)L  -  L'B'MA  -  A'MBL 

-  A'MA  +  A'MB(B'MB+Rf  2B'MA 
=  L'(B'MB  +  R)L  -  A'MBL  -  L'B'MA 

+  A'MBfB'MB  +  R)_1B'MA 

=  [L  -  (R+B'MB)"1  B'MA]'(B'MB+R)( L  -  (R  +  B'MBJ'WmA) 

>  0 
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Hence  Kr_j  £  ^k-1'  ®*nce  GQ  *G'  <  KR^  it  follows  from  equation 
(A. 8)  that  0  <  Kr_j  and  since  GQ  *G'  <  2r*j  5.  we  a^80  obtain 

GQ”  1G'  <  Q.E.D. 


Lemma  A.  5:  Consider  the  sequence  of  matrices  {£R}  generated  by  the 
equation 

2R1  =  F'(ER*  -  GQ~*G')~*F  +  ¥  +  L'RL  (A.  10) 

£n  =  ¥  (A.  1 1 ) 

where  R,  Q,  G,  F,  L  are  given  matrices  of  appropriate  dimension,  ¥ ,  R  and 

Q  are  positive  definite  and  symmetric  and  for  every  eigenvalue  X  =  a  +  bi  of 

2  2 

the  nxn  matrix  F  we  have  a  +  b  <1. 

Let  M  be  a  positive  definite  symmetric  matrix  such  that  F'MF  <  M, 
let  q  be  a  positive  scalar  such  that 

GQ^G'  <  qM"1  (A.  12) 

and  let  |x  be  a  positive  scalar  such  that 

F'MF  <  (1  -qn)M,  qn  <  1  (A.  13) 


The  existence  of  such  a  scalar  p  is  guaranteed  since  by  Lemma  A.  1  there 
exists  a  positive  scalar  v  such  that  VM  <  M  -  F'MF  and  any  scalar  ^  with 

v 

0  <  p.  <  —  satisfies  the  inequality  (A.  13).  Assume  further  that  the  matrices 
R  and  L  are  such  that 


¥+  L'RL  <  -y^-((l  -qn)M  -  F'MF)  (A.  14) 

Then  the  sequence  {£R}  converges  to  a  positive  definite  symmetric  matrix 
L  .  Furthermore  the  matrices  £,  are  positive  definite  and  for  all  k 

-GD  K 
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GQ^G'  <  ZRJ  ,  ZR  <  pM  (A.  15) 

Proof:  It  will  be  proved  under  our  assumptions  that 

0  <  Er  <  Zk  l  <  pM  (A.  16) 

then  by  Lemma  A.  3  convergence  will  follow  and  furthermore  the  inequality 
(A.  15)  will  be  satisfied  since  from  (A.  12)  and  (A.  13)  we  have 

GQ"*G'  <  qM’1  <  <  Z"1 

The  relation  (A.  16)  will  be  proved  by  induction.  We  have  from  (A.  14) 

0  <¥  =  <  pM  which  also  implies  (>Jf*  -  GQ~*G')-1  is  po  sitive  definite 

and  from  equation  (A.  10)  we  have  ¥  =  Z^  <  Z^  j.  Assume  that  ZR+j  <  ZR  < 
pM.  Then 

ZR  j  =  F'(ZR*  -GQ”*G')"*F  +  ¥  +  L'RL 

>  F'fZj^j  -GQ^G'^F  +  ¥  +  L'RL  =  Zk 

and  also 

ZR1  =  F*  j^1  ■GQ*1G')*1F  +  ¥  +  L'RL 

<  F't-^-M"1  -  GQ^O^F  +y+  L'RL 

<  F'f-jjj-M”1-  qM’1)  _IF +¥  +  L'RL 

=  -r-^—F'MF  +  ¥  +  L'RL  <  pM 
I  -  qp 

where  the  last  inequality  follows  from  relation  (A.  14).  Thus  we  obtain 
ZR  <  ZR  j  <  pM  and  the  induction  proof  is  complete  Q.  E.  D. 

We  are  now  ready  to  state  the  proof  of  Proposition  3.  1: 

Proof  of  Proposition  3.1:  It  is  required  to  prove  that  there  exists  a  scalar 
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Pj  such  that  for  every  P  with  0  <  P  <  Pj  there  exist  positive  scalars  a^bj 
such  that  for  every  at  b  with  0<  a  <  aj,  0<  b  <  bj  the  sequence  {K^} 
generated  by  the  equation 

Kk_j  -  A'[(l-p)Kk1--^GQ'1G'+-g-BR‘1B']"1A  +  a^  (A.17) 


K 


N 


a*. 


(A.  18) 


converges  to  a  positive  definite  symmetric  matrix  K  and  furthermore 
we  have 


jGQ^G'  <  K^1 


for  all  k 


(A.  19) 


The  Lemma  A.  3  will  be  used  to  reduce  the  proof  of  the  proposition  to 
proving  a  different  statement.  We  first  make  the  following  observatio  i: 
If  for  some  p,a,b  the  inequality  (A.  19)  holds  for  all  k  then 


K.  <  K.  . 
k  -  k-1 


for  all  k 


(A.  20) 


We  prove  this  fact  by  induction.  For  k  =  N  we  have  from  equation  (A.  17) 
a¥j=  Kn  <  Assume  Kk+j  5  Then  from  (A.  17) 

=  A'(  (l-P)K’1  --j^-GQ^G'  +  -g-  BRj"1  B')  "*A  +  ai[ 

>  A'((l-P)K‘|1--^GQ'1G'  +  -i-BR1‘IB'l‘1A  +  a^=  Kk 


K 


k-1 


or  K,  <  Kk.r 


Now  if  we  could  find  a  positive  definite  symmetric  matrix  S  such  that 


K,  <  S 
k  — 


for  all  k 


(A. 21) 


and  furthermore 


-jGQ_1G'  <  S’1 


(A.  22) 
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then  the  inequality  (A.  19)  would  be  satisfied  for  all  k  and  from  the  relations 
(A.  20)  and  (A.  21)  we  would  have  j  <  S  for  all  k.  This  in  turn 

would  imply  by  Lemma  A.  3  that  the  sequence  {K^}  converges  to  a  positive 
definite  symmetric  matrix  K _qq . 

Thus  in  order  to  prove  the  proposition  it  is  sufficient  to  demonstrate 
a  positive  scalar  P  j  <  1  and  for  every  p,  0  <  p  <  p  j  positive  scalars  a  j ,  b  j 
such  that  for  all  a,  b,  0  <  a  <  a^ ,  0<b<bj  there  exists  a  matrix  S  satisfying 
relations  (A.  21)  and  (A.  22). 

Since  the  pair  (A,  B)  is  stabilizable  there  exists  an  mxn  matrix  L 
such  that  the  matrix  (A-BL)  is  stable.  Let  Pj  be  a  scalar,  0  <  Pj  <  1  such 
that  the  matrix 

F,  =  (1  -  Pj)'1/2(A  -BL) 

is  also  stable.  Clearly  such  a  scalar  exists  and  for  every  p,  0  <  p  <  pj,  the 
matrix 

F  =  (1  -  P)’1/2(A  -  BL)  (A. 23) 

is  also  stable.  It  will  be  shown  below  that  Pj  satisfies  the  requirements  of 
the  proposition. 

Let  now  for  any  p,  0  <  p  <  p  A  =  (1  -  p )" 1  /2A,  B  -  (1-P)'1/2B. 
The  equation  (A.  17)  can  be  rewritten  as 

Kk  l  =  A'(  K’1  -  -g-GQ^G'  +-g-BRj1  B')  '  1 A  +  a^  (A. 24) 

and  since  the  matrix  F  of  equation  (A.  231  can  be  written  as  F  -  A  -  BL,  by 
using  Lemma  A.  4  we  obtain  that 


for  all  k 


(A. 25) 
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where  Z^  is  the  solution  of  the  equation 

Zk  i  =  F'fZ’1  - -p-GQ'^')"1?  +  a^  +  bL'RjL  (A. 26) 

ZN  =  (A.  27) 

provided  that  0  <  Z^  for  all  k  and 

-p- GQ” 1 G'  <  Zj^1  for  all  k.  (A. 28) 

Now  by  Lemma  A.  5  if 

a^+bL'RjL  <  [  (l-qp)M  -  F'MF)  (A.  29) 

where  M  is  a  positive  definite  symmetric  matrix  and  q,  p  are  positive  scalars 
such  that 


F'MF  <  M  (A.  30) 

-p-GQ_1G'  <  qM~ 1  (A.  31) 

F'MF  <  (1  -  qp)M  (A.  32) 

the  sequence  {Z^}  generated  by  equation  (A.  26)  converges  to  a  positive  defi¬ 
nite  symmetric  matrix  Z  ^  and  we  have  -p-GQ  *G'  <  Z^*  for  all  k  and 
Z^  <  yiM  for  all  k.  Thus  for  a,  b  satisfying  the  relation  (A.  29)  we  have  from 
(A.  25)  that  for  S  =  pM 

K,  <  S  for  all  k 

k 

Furthermore  since  0  <  qp  <  1  from  equation  (A.  31) 


-p-GQ^G*  <  qM'1  <  ^M'1  =  S_1 
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Therefore  for  S  =  p.M  the  relations  (A.  21)  and  (A.  22)  are  satisfied  and 
consequently  the  sequence  {K^}  converges  to  a  positive  definite  symmetric 
matrix  K  and  the  inequality  (A.  19)  is  satisfied  for  every  a,b  satisfying 
the  inequality  (A.  59).  It  is  clear  from  Lemma  A.  1  that  there  exist  positive 
scalars  a^ ,  bj  such  that  for  every  a,  b,  0  <  a  <  a  j ,  0<b<bj  the  inequality 
(A.  19)  is  satisfied.  Any  such  scalars  aj.bj  satisfy  the  requirements  of 
the  proposition.  Q.E.D. 

We  next  present  the  proof  of  Proposition  3.2. 

Proof  of  Proposition  3.2;  We  will  prove  that  for  the  state  of  the  closed- loop 
system 

xk+l  =  <A  "  BL^k  (A.  33) 

with 

L  =  (R  ♦  B'F  B)"1  B'F  A 
-00  -00 

and  for  any  positive  integer  N  we  have 

xNK-a>xN  L'RL,xk  <  xoK-ooxo  (A-  34) 

Then  from  the  positive  definiteness  of  (¥+  L'RL)  asymptotic  stability  of  the 
system  (A.  33)  follows. 

To  prove  the  relation  (A.  34)  we  will  use  the  following  identity  which 
is  familiar  from  Riccati  equation  theory.  This  identity  can  be  verified  in  a 
straight  forward  manner 

K  =  (A  -  BL)'F  (A  -  BL)  +  V  +  L'RL  (A.  35) 

-oo  -00 


We  also  have 
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F"1  =  (l-PHK'1  --^GQ^G')  <  K~*  -  ■tGQ"1G'  <  K'* 

-00  -oo  p  -  oo  p  —  -oo 


implying 


K  <  F 
-oo  -oo 


(A.  36) 


By  using  relations  (A.  35)  and  (A.  36)  we  now  have: 


xNK-ooxN  +  ^‘xk,'i'+L’BL>xk 

<  xNF-coxN  *  j'‘xk('l'+1-'RL>xk 
k=0 


N-2 


=  x*  ((A-BL)'F  (A-BL)  +^+  L'RL]xN  .+  Z  x'(t+L'RL)x, 
N-l  ‘®  k=  0  k 


N-2 


XN-  lK-ooxN-  1  +  ^  Qxk^+ L'RL^xk 


<  ^K^n-i  +x;(^L,RL)xo 

<  «;l(A-BU’FJ*-BL)t**L'RL|,o  =  «;K.<„xo 
or 

xNK-a,xN  +  ^  1  x^<V+L'RL)xk  <  x;Ka)xo 
k=  0 

Q.  E.  D. 


1 
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